AI generates covertly racist decisions about people based on their dialect.

Hundreds of millions of people now interact with language models, with uses ranging from help with writing1,2 to informing hiring decisions3. However, these language models are known to perpetuate systematic racial prejudices, making their judgements biased in problematic ways about groups such as African Americans4-7. Although previous research has focused on overt racism in language models, social scientists have argued that racism with a more subtle character has developed over time, particularly in the United States after the civil rights movement8,9. It is unknown whether this covert racism manifests in language models. Here, we demonstrate that language models embody covert racism in the form of dialect prejudice, exhibiting raciolinguistic stereotypes about speakers of African American English (AAE) that are more negative than any human stereotypes about African Americans ever experimentally recorded. By contrast, the language models' overt stereotypes about African Americans are more positive. Dialect prejudice has the potential for harmful consequences: language models are more likely to suggest that speakers of AAE be assigned less-prestigious jobs, be convicted of crimes and be sentenced to death. Finally, we show that current practices of alleviating racial bias in language models, such as human preference alignment, exacerbate the discrepancy between covert and overt stereotypes, by superficially obscuring the racism that language models maintain on a deeper level. Our findings have far-reaching implications for the fair and safe use of language technology.

Large language models in medical ethics: useful but not expert.

Large language models (LLMs) have now entered the realm of medical ethics. In a recent study, Balas et al examined the performance of GPT-4, a commercially available LLM, assessing its performance in generating responses to diverse medical ethics cases. Their findings reveal that GPT-4 demonstrates an ability to identify and articulate complex medical ethical issues, although its proficiency in encoding the depth of real-world ethical dilemmas remains an avenue for improvement. Investigating the integration of LLMs into medical ethics decision-making appears to be an interesting avenue of research. However, despite the promising trajectory of LLM technology in medicine, it is crucial to exercise caution and refrain from attributing their expertise to medical ethics. Our thesis follows an examination of the nature of expertise and the epistemic limitations that affect LLM technology. As a result, we propose two more fitting applications of LLMs in medical ethics: first, as tools for mining electronic health records or scientific literature, thereby supplementing evidence for resolving medical ethics cases, and second, as educational platforms to foster ethical reflection and critical thinking skills among students and residents. The integration of LLMs in medical ethics, while promising, requires careful consideration of their epistemic limitations. Consequently, a well-considered definition of their role in ethically sensitive decision-making is crucial.

As low as reasonably practicable (ALARP): a moral model for clinical risk management in the setting of technology dependence.

Children dependent on life-prolonging medical technology are often subject to a constant background risk of sudden death or catastrophic complications. Such children can be cared for in hospital, in an intensive care environment with highly trained nurses and doctors able to deliver specialised, life-saving care immediately. However, remaining in hospital, when life expectancy is limited, can considered to be a harm in of itself. Discharge home offers the possibility for an improved quality of life for the child and their family but comes with significant medical risks.When making decisions for children, two ethical models predominate, the promotion of the child's best interests or the avoidance of harm. However, in some circumstances, particularly for children with life-limiting and/or life-threatening illness, all options may be associated with risk. There are no good options, only potentially harmful choices.In this paper, we explore decisions made by one family in such circumstances. We describe a model adopted from risk management programmes beyond medicine, which offers a potential framework for identifying risks to the child that are morally permissible. Some risks and harms to a child, not ordinarily permitted, may be acceptable when undertaken in the pursuit of a specified desired good, so long as they are as low as reasonably practicable.

What it is like to be manic: a response to Director.

In a recent article, Director makes the case that many individuals with bipolar disorder have the capacity to consent to many decisions while acutely manic, even when those decisions are out of character and cause harm. Referring to recent qualitative evidence, I argue that Director overlooks a key mechanism of manic incapacity, an inflexible experience of the future that impairs one's ability to value. Without attention to the illness-specific experience of decision-making, capacity assessments risk false negatives in people with mania.

Reflexive control in emergency medicine.

Emergency physicians (EPs) navigate high-pressure environments, making rapid decisions amidst ambiguity. Their choices are informed by a complex interplay of experience, information, and external forces. While cognitive shortcuts (heuristics) expedite assessments, there are multiple ways they can be subtly manipulated, potentially leading to reflexive control: external actors steering EPs' decisions for their own benefit. Pharmaceutical companies, device manufacturers, and media narratives are among the numerous factors that influence the EPs' information landscape. Using tactics such as selective data dissemination, framing, and financial incentives, these actors can exploit pre-existing cognitive biases like anchoring, confirmation, and availability. This creates fertile ground for reflexive control, where EPs may believe they are acting independently while unknowingly serving the goals of external influencers. The consequences of manipulated decision making can be severe: misdiagnoses, inappropriate treatments, and increased healthcare costs. Ethical dilemmas arise when external pressures conflict with patient well-being. Recognizing these dangers empowers EPs to resist reflexive control through (1) critical thinking: examining information for potential biases and prioritizing evidence-based practices, (2) continuous education: learning about cognitive biases and mitigation strategies, and (3) institutional policies: implementing regulations to reduce external influence and to promote transparency. This vulnerability of emergency medicine decision making highlights the need for awareness, education, and robust ethical frameworks. Understanding reflexive control techniques is crucial for safeguarding patient care and promoting independent, ethical decision making in emergency medicine.

Pediatric Decision-Making: ethical aspects specific to neonates.

Recently published consensus recommendations on pediatric decision-making by Salter and colleagues (2023) did not address neonatal decision-making, due to the unique complexities of neonatal care. This essay explores three areas that impact neonatal decision-making: legal and policy considerations, rapid technological advancement, and the unique emotional burdens faced by parents and clinicians during the medical care of neonates. The authors evaluate the six consensus recommendations related to these considerations and conclude that the consensus recommendations apply to neonates.

Limits on Parental Discretion in Medical Decision-Making: pediatric intervention principles converge.

Pediatric intervention principles help clinicians and health-care institutions determine appropriate responses when parents' medical decisions place children at risk. Several intervention principles have been proposed and defended in the pediatric ethics literature. These principles may appear to provide conflicting guidance, but much of that conflict is superficial. First, seemingly different pediatric intervention principles sometimes converge on the same guidance. Second, these principles often aim to solve different problems in pediatrics or to operate in different background conditions. The potential for convergence between intervention principles-or at least an absence of conflict between them-matters for both the theory and practice of pediatric ethics. This article builds on the recent work of a diverse group of pediatric ethicists tasked with identifying consensus guidelines for pediatric decision-making.

Euthanasia and End-of-Life Decisions: From the Empirical Turn to Moral Intuitionism.

Most medical learned societies have endorsed both "equivalence" between all forms of withholding or withdrawing treatment and the "discontinuity" between euthanasia and practices to withhold or withdraw treatment. While the latter are morally acceptable insofar as they consist in letting the patient die, the former constitutes an illegitimate act of actively interfering with a patient's life. The moral distinction between killing and letting die has been hotly debated both conceptually and empirically, most notably by experimental philosophers, with inconclusive results. This article employs a "revisionary" intuititionist perspective to discuss the results of a clinical ethics study about intensivists' perceptions of withhold or withdraw decisions. The results show that practitioners' moral experience is at odds with both the discontinuity and equivalence theses. This outcome allows us to revisit certain concepts, such as intention and causal relationship, that are prominent in the conceptual debate. Intensivists also regard end-of-life decisions as being on a scale from least to most active, and whether they regard active forms of end-of-life decisions as ethically acceptable depends on the overarching professional values they endorse: the patient's best chances of survival, or the patient's quality of life.

Childhood Interests: what they are and why it matters.

This paper examines the concept and moral significance of "childhood interests." This concept is important in medical decision-making for children and more broadly in the field of pediatric ethics. The authors argue that childhood interests are identifiable components of childhood well-being that carry moral weight. Parents have a special role in protecting and promoting these interests and special obligations to do so. These parental obligations are grounded by the independent interests of the child, as well as the good of society more generally. Because parents have these child-rearing obligations, they must also have the authority and wide discretion necessary to fulfill them. However, while parental discretion is wide, it is not unlimited, for it must be used to safeguard and advance childhood interests.

A Personalized Patient Preference Predictor for Substituted Judgments in Healthcare: Technically Feasible and Ethically Desirable.

When making substituted judgments for incapacitated patients, surrogates often struggle to guess what the patient would want if they had capacity. Surrogates may also agonize over having the (sole) responsibility of making such a determination. To address such concerns, a Patient Preference Predictor (PPP) has been proposed that would use an algorithm to infer the treatment preferences of individual patients from population-level data about the known preferences of people with similar demographic characteristics. However, critics have suggested that even if such a PPP were more accurate, on average, than human surrogates in identifying patient preferences, the proposed algorithm would nevertheless fail to respect the patient's (former) autonomy since it draws on the 'wrong' kind of data: namely, data that are not specific to the individual patient and which therefore may not reflect their actual values, or their reasons for having the preferences they do. Taking such criticisms on board, we here propose a new approach: the Personalized Patient Preference Predictor (P4). The P4 is based on recent advances in machine learning, which allow technologies including large language models to be more cheaply and efficiently 'fine-tuned' on person-specific data. The P4, unlike the PPP, would be able to infer an individual patient's preferences from material (e.g., prior treatment decisions) that is in fact specific to them. Thus, we argue, in addition to being potentially more accurate at the individual level than the previously proposed PPP, the predictions of a P4 would also more directly reflect each patient's own reasons and values. In this article, we review recent discoveries in artificial intelligence research that suggest a P4 is technically feasible, and argue that, if it is developed and appropriately deployed, it should assuage some of the main autonomy-based concerns of critics of the original PPP. We then consider various objections to our proposal and offer some tentative replies.

What Are Humans Doing in the Loop? Co-Reasoning and Practical Judgment When Using Machine Learning-Driven Decision Aids.

Within the ethical debate on Machine Learning-driven decision support systems (ML_CDSS), notions such as "human in the loop" or "meaningful human control" are often cited as being necessary for ethical legitimacy. In addition, ethical principles usually serve as the major point of reference in ethical guidance documents, stating that conflicts between principles need to be weighed and balanced against each other. Starting from a neo-Kantian viewpoint inspired by Onora O'Neill, this article makes a concrete suggestion of how to interpret the role of the "human in the loop" and to overcome the perspective of rivaling ethical principles in the evaluation of AI in health care. We argue that patients should be perceived as "fellow workers" and epistemic partners in the interpretation of ML_CDSS outputs. We further highlight that a meaningful process of integrating (rather than weighing and balancing) ethical principles is most appropriate in the evaluation of medical AI.

Creating a safer and better functioning system: Lessons to be learned from the Netherlands for an ethical defence of an autonomy-only approach to assisted dying.

The proposal to allow assisted dying for people who are not severely ill reignited the Dutch end-of-life debate when it was submitted in 2016. A key criticism of this proposal is that it is too radical a departure from the safe and well-functioning system the Netherlands already has. The goal of this article is to respond to this criticism and question whether the Dutch system really can be described as safe and well functioning. I will reconsider the usefulness of the suffering criterion, and I will ultimately argue this criterion should be rejected altogether. Instead, we should consider moving towards an autonomy-only approach to assisted dying. This would resolve some significant issues occurring under the current system of assisted dying in the Netherlands and ultimately make the process safer and better functioning. I will then consider some possible objections to adopting an autonomy-only approach and provide some preliminary responses to these also. I will finally highlight some potential areas where further research may be necessary, namely, how to mitigate the effect of external factors such as poverty or other life aspects that may have the potential to distort the individual's ability to make autonomous decisions. I will also consider some possible international lessons that can be taken from both current as well as the proposed practice in the Netherlands.

On subjective measures of decision quality.

In times of person-centered care, it is all the more important to support patients in making good decisions about their care. One way to offer such support to patients is by way of Patient Decision Aids (PDAs). Ranging from patient brochures to web-based tools, PDAs explicitly state the decisions patients face, inform them about their medical options, help them to clarify and discuss their values, and ultimately make a decision. However, lingering discussions surround effectiveness research on PDAs. In this article, I focus on two subjective measures of decision quality that are widely used as outcome measures in effectiveness research on PDAs (i.e., the Decisional Conflict Scale (DCS) and measures of regret). Although these measurement instruments have attracted critical attention in the scientific literature, bioethicists have hardly engaged with them. Therefore, I set myself to analyze the relationship between (the different subscales of) the DCS and measures of regret, on the one hand, and ethical principles such as beneficence and autonomy, on the other hand. In light of that analysis, I will clarify some discussions regarding the use of these measures of decision quality in effectiveness research on PDAs. This should help us to align the way we evaluate PDAs with ethical principles and avoid that our attempts to support patients in making good decisions about their care that is so central to person-centered care point in unethical directions.

Ethical challenges in health care during collective hunger strikes in public or occupied spaces.

Public collective hunger strikes take place in complex social and political contexts, require medical attention and present ethical challenges to physicians. Empirical research, the ethical debate to date and existing guidelines by the World Medical Association focus almost exclusively on hunger strikes in detention. However, the public space differs substantially with regard to the conditions for the provision of health care and the diverse groups of healthcare providers or stakeholders involved. By reviewing empirical research on the experience of health professionals with public collective hunger strikes, we identified the following ethical challenges: (1) establishment of a trustful physician-striker relationship, (2) balancing of medico-ethical principles in medical decision-making, (3) handling of loyalty conflicts and (4) preservation of professional independence and the risk of political instrumentalization. Some of these challenges have already been described and discussed, yet not contextualized for public collective strikes, while others are novel. The presence of voluntary physicians may offer opportunities for a trustful relationship and, hence, for ethical treatment decisions. According to our findings, it requires more attention to how to realise autonomous medical decisions in the complex context of a dynamic, often unstructured and politically charged setting, which ethical norms should shape the professional role of voluntary physicians, and what is the influence of the hunger strikers' collective on individual healthcare decisions. Our article can serve as a starting point for further ethical discussion. It can also provide the basis for the development of potential guidelines to support health professionals involved in public collective hunger strikes.

Arguments for a ban on pediatric intersex surgery: A dis/analogy with Jehovah witness blood transfusion.

This article argues for a ban on the performance of medically unnecessary genital normalizing surgeries as part of assigning a binary sex/gender to infants with intersex conditions on the basis of autonomy, regardless of etiology. It does this via a dis/analogy with the classic case in bioethics of Jehovah Witness (JW) parents' inability to refuse life-saving blood transfusions for their minor children. Both cases address ethical medical practice in situations where parents are making irreversible medical decisions on the basis of values strongly held, identity, and relationship-shaping values-such as religious beliefs or beliefs regarding the inherent value of binary sex/gender-amidst ethical pluralism. Furthermore, it takes seriously-as we must in the intersex case-that the restriction of parents' right to choose will likely result in serious harms to potentially large percentage of patients, their families, and their larger communities. I address the objection that parents' capacity to choose is restricted in the JW case on the basis of the harm principle or a duty to nonmaleficence, given that the result of parent choice would be death. I provide evidence that this is mistaken from how we treat epistemic uncertainty in the JW case and from cases in which clinicians are ethically obligated to restrict the autonomy of nonminor patients. I conclude that we restrict the parents' right to choose in the JW case-and should in the case of pediatric intersex surgery-to secure patient's future autonomy.

Ethical and practical considerations of pediatric refusal in clinical anesthesia: An educational review.

Children commonly refuse induction of anesthesia. Anesthesia providers must then decide whether to honor the child's dissent or to proceed over objection. In some circumstances, a forced induction involves restraining the child, incurring both practical and ethical harms to the patient-provider encounter. This educational review explores the practical dilemma encountered when a child dissents to induction of anesthesia. In the course of exploring this dilemma, dissent and associated terms are defined and compared, and the prominent ethical underpinnings regarding pediatric decision-making are described to clarify dissent as an ethical and practical concept. Important legal and professional standards are summarized, and practice trends are discussed to depict the current state of practice, including novel approaches to honoring pediatric dissent for elective surgeries. This information is then used to invite providers to consider where they ethically situate themselves within a legally and professionally defined space of acceptable practice. Finally, these considerations are synthesized to discuss important nuances regarding pediatric refusal, and some key questions are presented for clinicians to ponder as they consider their practice of choosing whether to honor pediatric dissent at induction.

Evaluation of an interactive education workshop on hospital pharmacists' ethical reasoning: an observational study.

BACKGROUND: Pharmacists are often faced with scenarios in practice that require application of ethical reasoning and decision-making skills. There is limited research on the ethical decision-making processes of hospital pharmacists. Pharmacists who are compassionate and put the interests of their patients first are thought to positively impact on patient care, but there are often complex health-care system pressures in the hospital setting that cause pharmacists to behave in ways that may conflict with professional values and behaviours. This multisite study aimed to evaluate an interactive education workshop on hospital pharmacists' ethical reasoning skills and explore the need for ongoing training and support. METHODS: This mixed-methods study was carried out across two health services including three hospitals. It incorporated a pre-workshop survey, a feedback survey immediately post-workshop and a third survey four weeks after the workshop. Semi-structured interviews were conducted with hospital pharmacists at least four weeks after the ethics workshop. RESULTS: In total, 32 participants completed the pre-workshop survey, nominating peers/colleagues as the most common source of support they would consult to inform ethical decision-making (17/118 sources of support). Almost all (n = 31/33; 94%) strongly agreed/agreed that the education session provided them with ethical reasoning skills and a process/framework which they could use when faced with an ethical issue. Pre- and post-survey responses showed increased self-confidence in identifying the regulatory frameworks applicable to pharmacy privacy requirements (p = 0.011) and ethical issues applicable to pharmacy privacy requirements (p = 0.002), as well as applying ethical reasoning to scenarios that involve pharmacy privacy dilemmas/issues (p = 0.004). Participants' self confidence in knowing where to find support when faced with clinical and non-clinical ethics questions was improved (p = 0.002 and p = 0.003 respectively). Participants supported the introduction of quarterly ethics cafes after the workshop, compared to before the workshop (p = 0.001). CONCLUSION: Hospital pharmacists rely on discussions with colleagues to brainstorm how to address ethical issues. This study showed that a targeted interactive education workshop facilitated familiarity with ethics resources and decision-making processes. It also demonstrated that this approach could be used to enhance hospital pharmacists' readiness, confidence, and capabilities to recognise and respond to challenging ethical issues.

Assessment of decision-making autonomy in chronic pain patients: a pilot study.

BACKGROUND: Patient decision-making autonomy refers to the patients' ability to freely exert their own choices and make their own decisions, given sufficient resources and information to do so. In pain medicine, it is accepted that appropriate beneficial management aims to propose an individualized treatment plan shared with the patients, as agents, to help them live as autonomously as possible with their pain. However, are patients in chronic pain centers sufficiently autonomous to participate in the therapeutic decisions that concern them? As this question still remains unanswered, a pilot study was set up to that aim. METHODS: Over a 2-month period, first-time patients within a tertiary multidisciplinary pain center underwent a systematic evaluation of their autonomy using the MacArthur Competence Assessment Tool for Treatment (MacCAT-T), considered the benchmark tool for measuring a patient's ability to consent to treatment. Demographic data and pain characteristics of the patients were collected and their respective attending pain physicians were asked to clinically assess their patients' degree of autonomy. Another physician, who had not participated in the initial patient evaluation, subsequently administered the MacCAT-T questionnaire to the same patients. RESULTS: Twenty-seven patients were included during the study period (21 women and 6 men), with an average age of 50 years. The average duration of pain was 8 years. Based on their clinical experience, the 4 different pain physicians in charge of these patients considered that out of 25 assessed patients, 22 of them (89%) had full decision-making capacity, with no deficit in autonomy. According to the MacCAT-T results, only 13 of these 25 patients (48%) had no deficit, while 7 (26%) had a major deficit in autonomy. The only patient characteristic that appeared to be related to autonomy was pain type, specifically nociplastic pain. The average time taken to complete the test was 20 min, and patients were very satisfied with the interview. CONCLUSION: Results from the present pilot study suggest that patients suffering from chronic pain do not appear to be entirely autonomous in their decision to consent to the proposed treatment plan according to the MacCAT-T questionnaire, and physicians seem to find it difficult to properly assess this competence in a clinical setting. Further studies with larger samples are needed to better evaluate this concept to improve the complex management of these patients.

Artificial Intelligence to support ethical decision-making for incapacitated patients: a survey among German anesthesiologists and internists.

BACKGROUND: Artificial intelligence (AI) has revolutionized various healthcare domains, where AI algorithms sometimes even outperform human specialists. However, the field of clinical ethics has remained largely untouched by AI advances. This study explores the attitudes of anesthesiologists and internists towards the use of AI-driven preference prediction tools to support ethical decision-making for incapacitated patients. METHODS: A questionnaire was developed and pretested among medical students. The questionnaire was distributed to 200 German anesthesiologists and 200 German internists, thereby focusing on physicians who often encounter patients lacking decision-making capacity. The questionnaire covered attitudes toward AI-driven preference prediction, availability and utilization of Clinical Ethics Support Services (CESS), and experiences with ethically challenging situations. Descriptive statistics and bivariate analysis was performed. Qualitative responses were analyzed using content analysis in a mixed inductive-deductive approach. RESULTS: Participants were predominantly male (69.3%), with ages ranging from 27 to 77. Most worked in nonacademic hospitals (82%). Physicians generally showed hesitance toward AI-driven preference prediction, citing concerns about the loss of individuality and humanity, lack of explicability in AI results, and doubts about AI's ability to encompass the ethical deliberation process. In contrast, physicians had a more positive opinion of CESS. Availability of CESS varied, with 81.8% of participants reporting access. Among those without access, 91.8% expressed a desire for CESS. Physicians' reluctance toward AI-driven preference prediction aligns with concerns about transparency, individuality, and human-machine interaction. While AI could enhance the accuracy of predictions and reduce surrogate burden, concerns about potential biases, de-humanisation, and lack of explicability persist. CONCLUSIONS: German physicians frequently encountering incapacitated patients exhibit hesitance toward AI-driven preference prediction but hold a higher esteem for CESS. Addressing concerns about individuality, explicability, and human-machine roles may facilitate the acceptance of AI in clinical ethics. Further research into patient and surrogate perspectives is needed to ensure AI aligns with patient preferences and values in complex medical decisions.

How prehospital emergency personnel manage ethical challenges: the importance of confidence, trust, and safety.

BACKGROUND: Ethical challenges constitute an inseparable part of daily decision-making processes in all areas of healthcare. Ethical challenges are associated with moral distress that can lead to burnout. Clinical ethics support has proven useful to address and manage such challenges. This paper explores how prehospital emergency personnel manage ethical challenges. The study is part of a larger action research project to develop and test an approach to clinical ethics support that is sensitive to the context of emergency medicine. METHODS: We explored ethical challenges and management strategies in three focus groups, with 15 participants in total, each attended by emergency medical technicians, paramedics, and prehospital anaesthesiologists. Focus groups were audio-recorded and transcribed verbatim. The approach to data analysis was systematic text condensation approach. RESULTS: We stratified the management of ethical challenges into actions before, during, and after incidents. Before incidents, participants stressed the importance of mutual understandings, shared worldviews, and a supportive approach to managing emotions. During an incident, the participants employed moral perception, moral judgments, and moral actions. After an incident, the participants described sharing ethical challenges only to a limited extent as sharing was emotionally challenging, and not actively supported by workplace culture, or organisational procedures. The participants primarily managed ethical challenges informally, often using humour to cope. CONCLUSION: Our analysis supports and clarifies that confidence, trust, and safety in relation to colleagues, management, and the wider organisation are essential for prehospital emergency personnel to share ethical challenges and preventing moral distress turning into burnout.