Powerful microscopy technologies decode spatially organized cellular networks that drive response to immunotherapy in humans.

In tumors, immune cells organize into networks of different sizes and composition, including complex tertiary lymphoid structures and recently identified networks centered around the chemokines CXCL9/10/11 and CCL19. New commercially available highly multiplexed microscopy using cyclical RNA in situ hybridization and antibody-based approaches have the potential to establish the organization of the immune response in human tissue and serve as a foundation for future immunology research.

Large Language Model Influence on Management Reasoning: A Randomized Controlled Trial.

Importance: Large language model (LLM) artificial intelligence (AI) systems have shown promise in diagnostic reasoning, but their utility in management reasoning with no clear right answers is unknown. Objective: To determine whether LLM assistance improves physician performance on open-ended management reasoning tasks compared to conventional resources. Design: Prospective, randomized controlled trial conducted from 30 November 2023 to 21 April 2024. Setting: Multi-institutional study from Stanford University, Beth Israel Deaconess Medical Center, and the University of Virginia involving physicians from across the United States. Participants: 92 practicing attending physicians and residents with training in internal medicine, family medicine, or emergency medicine. Intervention: Five expert-developed clinical case vignettes were presented with multiple open-ended management questions and scoring rubrics created through a Delphi process. Physicians were randomized to use either GPT-4 via ChatGPT Plus in addition to conventional resources (e.g., UpToDate, Google), or conventional resources alone. Main Outcomes and Measures: The primary outcome was difference in total score between groups on expert-developed scoring rubrics. Secondary outcomes included domain-specific scores and time spent per case. Results: Physicians using the LLM scored higher compared to those using conventional resources (mean difference 6.5 %, 95% CI 2.7-10.2, p<0.001). Significant improvements were seen in management decisions (6.1%, 95% CI 2.5-9.7, p=0.001), diagnostic decisions (12.1%, 95% CI 3.1-21.0, p=0.009), and case-specific (6.2%, 95% CI 2.4-9.9, p=0.002) domains. GPT-4 users spent more time per case (mean difference 119.3 seconds, 95% CI 17.4-221.2, p=0.02). There was no significant difference between GPT-4-augmented physicians and GPT-4 alone (-0.9%, 95% CI -9.0 to 7.2, p=0.8). Conclusions and Relevance: LLM assistance improved physician management reasoning compared to conventional resources, with particular gains in contextual and patient-specific decision-making. These findings indicate that LLMs can augment management decision-making in complex cases. Trial registration: ClinicalTrials.gov Identifier: NCT06208423; https://classic.clinicaltrials.gov/ct2/show/NCT06208423.

Pathophysiological Features in Electronic Medical Records Sustain Model Performance under Temporal Dataset Shift.

Access to real-world data streams like electronic medical records (EMRs) has accelerated the development of supervised machine learning (ML) models for clinical applications. However, few studies investigate the differential impact of particular features in the EMR on model performance under temporal dataset shift. To explain how features in the EMR impact models over time, this study aggregates features into feature groups by their source (e.g. medication orders, diagnosis codes and lab results) and feature categories based on their reflection of patient pathophysiology or healthcare processes. We adapt Shapley values to explain feature groups' and feature categories' marginal contribution to initial and sustained model performance. We investigate three standard clinical prediction tasks and find that while feature contributions to initial performance differ across tasks, pathophysiological features help mitigate temporal discrimination deterioration. These results provide interpretable insights on how specific feature groups contribute to model performance and robustness to temporal dataset shift.

Electronic Phenotyping of Urinary Tract Infections as a Silver Standard Label for Machine Learning.

This study explored the efficacy of electronic phenotyping in data labeling for machine learning with a focus on urinary tract infections (UTIs). We contrasted labels from electronic phenotyping against previously published labels such as urine culture positivity. In comparison, electronic phenotyping showed the potential to enhance specificity in UTI labeling while maintaining similar sensitivity and was easily scaled for application to a large dataset suitable for machine learning, which we used to train and validate a machine learning model. Electronic phenotyping offers a valuable method for machine learning label generation in healthcare, with potential benefits for patient care and antimicrobial stewardship. Further research will expand its application and optimize techniques for increased performance.

Predictability of buprenorphine-naloxone treatment retention: A multi-site analysis combining electronic health records and machine learning.

BACKGROUND AND AIMS: Opioid use disorder (OUD) and opioid dependence lead to significant morbidity and mortality, yet treatment retention, crucial for the effectiveness of medications like buprenorphine-naloxone, remains unpredictable. Our objective was to determine the predictability of 6-month retention in buprenorphine-naloxone treatment using electronic health record (EHR) data from diverse clinical settings and to identify key predictors. DESIGN: This retrospective observational study developed and validated machine learning-based clinical risk prediction models using EHR data. SETTING AND CASES: Data were sourced from Stanford University's healthcare system and Holmusk's NeuroBlu database, reflecting a wide range of healthcare settings. The study analyzed 1800 Stanford and 7957 NeuroBlu treatment encounters from 2008 to 2023 and from 2003 to 2023, respectively. MEASUREMENTS: Predict continuous prescription of buprenorphine-naloxone for at least 6 months, without a gap of more than 30 days. The performance of machine learning prediction models was assessed by area under receiver operating characteristic (ROC-AUC) analysis as well as precision, recall and calibration. To further validate our approach's clinical applicability, we conducted two secondary analyses: a time-to-event analysis on a single site to estimate the duration of buprenorphine-naloxone treatment continuity evaluated by the C-index and a comparative evaluation against predictions made by three human clinical experts. FINDINGS: Attrition rates at 6 months were 58% (NeuroBlu) and 61% (Stanford). Prediction models trained and internally validated on NeuroBlu data achieved ROC-AUCs up to 75.8 (95% confidence interval [CI] = 73.6-78.0). Addiction medicine specialists' predictions show a ROC-AUC of 67.8 (95% CI = 50.4-85.2). Time-to-event analysis on Stanford data indicated a median treatment retention time of 65 days, with random survival forest model achieving an average C-index of 65.9. The top predictor of treatment retention identified included the diagnosis of opioid dependence. CONCLUSIONS: US patients with opioid use disorder or opioid dependence treated with buprenorphine-naloxone prescriptions appear to have a high (∼60%) treatment attrition by 6 months. Machine learning models trained on diverse electronic health record datasets appear to be able to predict treatment continuity with accuracy comparable to that of clinical experts.

Socioeconomic Disparities and the Prevalence of Antimicrobial Resistance.

BACKGROUND: The increased prevalence of antimicrobial resistant (AMR) infections is a significant global health threat, resulting in increased morbidity, mortality, and costs. The drivers of AMR are complex and potentially impacted by socioeconomic factors. We investigated the relationships between geographic and socioeconomic factors and AMR. METHODS: We collected select patient bacterial culture results from 2015 to 2020 from electronic health records (EHR) of two expansive healthcare systems within the Dallas-Fort Worth, TX (DFW) metropolitan area. Among individuals with EHR records who resided in the four most populus counties in DFW, culture data were aggregated. Case counts for each organism studied were standardized per 1,000 persons per area population. Using residential addresses, the cultures were geocoded and linked to socioeconomic index values. Spatial autocorrelation tests identified geographic clusters of high and low AMR organism prevalence and correlations with established socioeconomic indices. RESULTS: We found significant clusters of AMR organisms in areas with high levels of deprivation, as measured by the Area Deprivation Index (ADI). We found a significant spatial autocorrelation between ADI and the prevalence of AMR organisms, particularly for AmpC and MRSA with 14% and 13%, respectively, of the variability in prevalence rates being attributable to their relationship with the ADI values of the neighboring locations. CONCLUSIONS: We found that areas with a high ADI are more likely to have higher rates of AMR organisms. Interventions that improve socioeconomic factors such as poverty, unemployment, decreased access to healthcare, crowding, and sanitation in these areas of high prevalence may reduce the spread of AMR.

Single-cell transcriptomic analyses reveal distinct immune cell contributions to epithelial barrier dysfunction in checkpoint inhibitor colitis.

Immune checkpoint inhibitor (ICI) therapy has revolutionized oncology, but treatments are limited by immune-related adverse events, including checkpoint inhibitor colitis (irColitis). Little is understood about the pathogenic mechanisms driving irColitis, which does not readily occur in model organisms, such as mice. To define molecular drivers of irColitis, we used single-cell multi-omics to profile approximately 300,000 cells from the colon mucosa and blood of 13 patients with cancer who developed irColitis (nine on anti-PD-1 or anti-CTLA-4 monotherapy and four on dual ICI therapy; most patients had skin or lung cancer), eight controls on ICI therapy and eight healthy controls. Patients with irColitis showed expanded mucosal Tregs, ITGAEHi CD8 tissue-resident memory T cells expressing CXCL13 and Th17 gene programs and recirculating ITGB2Hi CD8 T cells. Cytotoxic GNLYHi CD4 T cells, recirculating ITGB2Hi CD8 T cells and endothelial cells expressing hypoxia gene programs were further expanded in colitis associated with anti-PD-1/CTLA-4 therapy compared to anti-PD-1 therapy. Luminal epithelial cells in patients with irColitis expressed PCSK9, PD-L1 and interferon-induced signatures associated with apoptosis, increased cell turnover and malabsorption. Together, these data suggest roles for circulating T cells and epithelial-immune crosstalk critical to PD-1/CTLA-4-dependent tolerance and barrier function and identify potential therapeutic targets for irColitis.

Feedback Loop Failure Modes in Medical Diagnosis: How Biases Can Emerge and Be Reinforced.

BACKGROUND: Medical diagnosis in practice connects to research through continuous feedback loops: Studies of diagnosed cases shape our understanding of disease, which shapes future diagnostic practice. Without accounting for an imperfect and complex diagnostic process in which some cases are more likely to be diagnosed correctly (or diagnosed at all), the feedback loop can inadvertently exacerbate future diagnostic errors and biases. FRAMEWORK: A feedback loop failure occurs if misleading evidence about disease etiology encourages systematic errors that self-perpetuate, compromising future diagnoses and patient care. This article defines scenarios for feedback loop failure in medical diagnosis. DESIGN: Through simulated cases, we characterize how disease incidence, presentation, and risk factors can be misunderstood when observational data are summarized naive to biases arising from diagnostic error. A fourth simulation extends to a progressive disease. RESULTS: When severe cases of a disease are diagnosed more readily, less severe cases go undiagnosed, increasingly leading to underestimation of the prevalence and heterogeneity of the disease presentation. Observed differences in incidence and symptoms between demographic groups may be driven by differences in risk, presentation, the diagnostic process itself, or a combination of these. We suggested how perceptions about risk factors and representativeness may drive the likelihood of diagnosis. Differing diagnosis rates between patient groups can feed back to increasingly greater diagnostic errors and disparities in the timing of diagnosis and treatment. CONCLUSIONS: A feedback loop between past data and future medical practice may seem obviously beneficial. However, under plausible scenarios, poorly implemented feedback loops can degrade care. Direct summaries from observational data based on diagnosed individuals may be misleading, especially concerning those symptoms and risk factors that influence the diagnostic process itself. HIGHLIGHTS: Current evidence about a disease can (and should) influence the diagnostic process. A feedback loop failure may occur if biased "evidence" encourages diagnostic errors, leading to future errors in the evidence base.When diagnostic accuracy varies for mild versus severe cases or between demographic groups, incorrect conclusions about disease prevalence and presentation will result without specifically accounting for such variability.Use of demographic characteristics in the diagnostic process should be done with careful justification, in particular avoiding potential cognitive biases and overcorrection.

Influence of a Large Language Model on Diagnostic Reasoning: A Randomized Clinical Vignette Study.

Importance: Diagnostic errors are common and cause significant morbidity. Large language models (LLMs) have shown promise in their performance on both multiple-choice and open-ended medical reasoning examinations, but it remains unknown whether the use of such tools improves diagnostic reasoning. Objective: To assess the impact of the GPT-4 LLM on physicians' diagnostic reasoning compared to conventional resources. Design: Multi-center, randomized clinical vignette study. Setting: The study was conducted using remote video conferencing with physicians across the country and in-person participation across multiple academic medical institutions. Participants: Resident and attending physicians with training in family medicine, internal medicine, or emergency medicine. Interventions: Participants were randomized to access GPT-4 in addition to conventional diagnostic resources or to just conventional resources. They were allocated 60 minutes to review up to six clinical vignettes adapted from established diagnostic reasoning exams. Main Outcomes and Measures: The primary outcome was diagnostic performance based on differential diagnosis accuracy, appropriateness of supporting and opposing factors, and next diagnostic evaluation steps. Secondary outcomes included time spent per case and final diagnosis. Results: 50 physicians (26 attendings, 24 residents) participated, with an average of 5.2 cases completed per participant. The median diagnostic reasoning score per case was 76.3 percent (IQR 65.8 to 86.8) for the GPT-4 group and 73.7 percent (IQR 63.2 to 84.2) for the conventional resources group, with an adjusted difference of 1.6 percentage points (95% CI -4.4 to 7.6; p=0.60). The median time spent on cases for the GPT-4 group was 519 seconds (IQR 371 to 668 seconds), compared to 565 seconds (IQR 456 to 788 seconds) for the conventional resources group, with a time difference of -82 seconds (95% CI -195 to 31; p=0.20). GPT-4 alone scored 15.5 percentage points (95% CI 1.5 to 29, p=0.03) higher than the conventional resources group. Conclusions and Relevance: In a clinical vignette-based study, the availability of GPT-4 to physicians as a diagnostic aid did not significantly improve clinical reasoning compared to conventional resources, although it may improve components of clinical reasoning such as efficiency. GPT-4 alone demonstrated higher performance than both physician groups, suggesting opportunities for further improvement in physician-AI collaboration in clinical practice.

Human lung cancer harbors spatially organized stem-immunity hubs associated with response to immunotherapy.

The organization of immune cells in human tumors is not well understood. Immunogenic tumors harbor spatially localized multicellular 'immunity hubs' defined by expression of the T cell-attracting chemokines CXCL10/CXCL11 and abundant T cells. Here, we examined immunity hubs in human pre-immunotherapy lung cancer specimens and found an association with beneficial response to PD-1 blockade. Critically, we discovered the stem-immunity hub, a subtype of immunity hub strongly associated with favorable PD-1-blockade outcome. This hub is distinct from mature tertiary lymphoid structures and is enriched for stem-like TCF7+PD-1+CD8+ T cells, activated CCR7+LAMP3+ dendritic cells and CCL19+ fibroblasts as well as chemokines that organize these cells. Within the stem-immunity hub, we find preferential interactions between CXCL10+ macrophages and TCF7-CD8+ T cells as well as between mature regulatory dendritic cells and TCF7+CD4+ and regulatory T cells. These results provide a picture of the spatial organization of the human intratumoral immune response and its relevance to patient immunotherapy outcomes.

Diagnostic reasoning prompts reveal the potential for large language model interpretability in medicine.

One of the major barriers to using large language models (LLMs) in medicine is the perception they use uninterpretable methods to make clinical decisions that are inherently different from the cognitive processes of clinicians. In this manuscript we develop diagnostic reasoning prompts to study whether LLMs can imitate clinical reasoning while accurately forming a diagnosis. We find that GPT-4 can be prompted to mimic the common clinical reasoning processes of clinicians without sacrificing diagnostic accuracy. This is significant because an LLM that can imitate clinical reasoning to provide an interpretable rationale offers physicians a means to evaluate whether an LLMs response is likely correct and can be trusted for patient care. Prompting methods that use diagnostic reasoning have the potential to mitigate the "black box" limitations of LLMs, bringing them one step closer to safe and effective use in medicine.

Investigation of air dispersal during a rhinovirus outbreak in a pediatric intensive care unit.

BACKGROUND: While airborne transmission of rhinovirus is recognized in indoor settings, its role in hospital transmission remains unclear. METHODS: We investigated an outbreak of rhinovirus in a pediatric intensive care unit (PICU) to assess air dispersal. We collected clinical, environmental, and air samples, and staff's surgical masks for viral load and phylogenetic analysis. Hand hygiene compliance and the number of air changes per hour in the PICU were measured. A case-control analysis was performed to identify nosocomial rhinovirus risk factors. RESULTS: Between March 31, 2023, and April 2, 2023, three patients acquired rhinovirus in a cubicle (air changes per hour: 14) of 12-bed PICU. A portable air-cleaning unit was placed promptly. Air samples (72,000 L in 6 hours) from the cohort area, and outer surfaces of staff's masks (n = 8), were rhinovirus RNA-negative. Hand hygiene compliance showed no significant differences (31/34, 91.2% vs 33/37, 89.2%, P = 1) before and during outbreak. Only 1 environmental sample (3.8%) was positive (1.86 × 103 copies/mL). Case-control and next-generation sequencing analysis implicated an infected staff member as the source. CONCLUSIONS: Our findings suggest that air dispersal of rhinovirus was not documented in the well-ventilated PICU during the outbreak. Further research is needed to better understand the dynamics of rhinovirus transmission in health care settings.

Electrocardiographic deep learning for predicting post-procedural mortality: a model development and validation study.

BACKGROUND: Preoperative risk assessments used in clinical practice are insufficient in their ability to identify risk for postoperative mortality. Deep-learning analysis of electrocardiography can identify hidden risk markers that can help to prognosticate postoperative mortality. We aimed to develop a prognostic model that accurately predicts postoperative mortality in patients undergoing medical procedures and who had received preoperative electrocardiographic diagnostic testing. METHODS: In a derivation cohort of preoperative patients with available electrocardiograms (ECGs) from Cedars-Sinai Medical Center (Los Angeles, CA, USA) between Jan 1, 2015 and Dec 31, 2019, a deep-learning algorithm was developed to leverage waveform signals to discriminate postoperative mortality. We randomly split patients (8:1:1) into subsets for training, internal validation, and final algorithm test analyses. Model performance was assessed using area under the receiver operating characteristic curve (AUC) values in the hold-out test dataset and in two external hospital cohorts and compared with the established Revised Cardiac Risk Index (RCRI) score. The primary outcome was post-procedural mortality across three health-care systems. FINDINGS: 45 969 patients had a complete ECG waveform image available for at least one 12-lead ECG performed within the 30 days before the procedure date (59 975 inpatient procedures and 112 794 ECGs): 36 839 patients in the training dataset, 4549 in the internal validation dataset, and 4581 in the internal test dataset. In the held-out internal test cohort, the algorithm discriminates mortality with an AUC value of 0·83 (95% CI 0·79-0·87), surpassing the discrimination of the RCRI score with an AUC of 0·67 (0·61-0·72). The algorithm similarly discriminated risk for mortality in two independent US health-care systems, with AUCs of 0·79 (0·75-0·83) and 0·75 (0·74-0·76), respectively. Patients determined to be high risk by the deep-learning model had an unadjusted odds ratio (OR) of 8·83 (5·57-13·20) for postoperative mortality compared with an unadjusted OR of 2·08 (0·77-3·50) for postoperative mortality for RCRI scores of more than 2. The deep-learning algorithm performed similarly for patients undergoing cardiac surgery (AUC 0·85 [0·77-0·92]), non-cardiac surgery (AUC 0·83 [0·79-0·88]), and catheterisation or endoscopy suite procedures (AUC 0·76 [0·72-0·81]). INTERPRETATION: A deep-learning algorithm interpreting preoperative ECGs can improve discrimination of postoperative mortality. The deep-learning algorithm worked equally well for risk stratification of cardiac surgeries, non-cardiac surgeries, and catheterisation laboratory procedures, and was validated in three independent health-care systems. This algorithm can provide additional information to clinicians making the decision to perform medical procedures and stratify the risk of future complications. FUNDING: National Heart, Lung, and Blood Institute.

Single-cell atlas of the liver myeloid compartment before and after cure of chronic viral hepatitis.

BACKGROUND & AIMS: Chronic viral infections present serious public health challenges; however, direct-acting antivirals (DAAs) are now able to cure nearly all patients infected with hepatitis C virus (HCV), representing the only cure of a human chronic viral infection to date. DAAs provide a valuable opportunity to study immune pathways in the reversal of chronic immune failures in an in vivo human system. METHODS: To leverage this opportunity, we used plate-based single-cell RNA-seq to deeply profile myeloid cells from liver fine needle aspirates in patients with HCV before and after DAA treatment. We comprehensively characterised liver neutrophils, eosinophils, mast cells, conventional dendritic cells, plasmacytoid dendritic cells, classical monocytes, non-classical monocytes, and macrophages, and defined fine-grained subpopulations of several cell types. RESULTS: We discovered cell type-specific changes post-cure, including an increase in MCM7+STMN1+ proliferating CD1C+ conventional dendritic cells, which may support restoration from chronic exhaustion. We observed an expected downregulation of interferon-stimulated genes (ISGs) post-cure as well as an unexpected inverse relationship between pre-treatment viral load and post-cure ISG expression in each cell type, revealing a link between viral loads and sustained modifications of the host's immune system. We found an upregulation of PD-L1/L2 gene expression in ISG-high neutrophils and IDO1 expression in eosinophils, pinpointing cell subpopulations crucial for immune regulation. We identified three recurring gene programmes shared by multiple cell types, distilling core functions of the myeloid compartment. CONCLUSIONS: This comprehensive single-cell RNA-seq atlas of human liver myeloid cells in response to cure of chronic viral infections reveals principles of liver immunity and provides immunotherapeutic insights. CLINICAL TRIAL REGISTRATION: This study is registered at ClinicalTrials.gov (NCT02476617). IMPACT AND IMPLICATIONS: Chronic viral liver infections continue to be a major public health problem. Single-cell characterisation of liver immune cells during hepatitis C and post-cure provides unique insights into the architecture of liver immunity contributing to the resolution of the first curable chronic viral infection of humans. Multiple layers of innate immune regulation during chronic infections and persistent immune modifications after cure are revealed. Researchers and clinicians may leverage these findings to develop methods to optimise the post-cure environment for HCV and develop novel therapeutic approaches for other chronic viral infections.

Session Introduction: Artificial Intelligence in Clinical Medicine: Generative and Interactive Systems at the Human-Machine Interface.

Artificial Intelligence (AI) models are substantially enhancing the capability to analyze complex and multi-dimensional datasets. Generative AI and deep learning models have demonstrated significant advancements in extracting knowledge from unstructured text, imaging as well as structured and tabular data. This recent breakthrough in AI has inspired research in medicine, leading to the development of numerous tools for creating clinical decision support systems, monitoring tools, image interpretation, and triaging capabilities. Nevertheless, comprehensive research is imperative to evaluate the potential impact and implications of AI systems in healthcare. At the 2024 Pacific Symposium on Biocomputing (PSB) session entitled "Artificial Intelligence in Clinical Medicine: Generative and Interactive Systems at the Human-Machine Interface", we spotlight research that develops and applies AI algorithms to solve real-world problems in healthcare.

The Problem of Pain in Rheumatology: Variations in Case Definitions Derived From Chronic Pain Phenotyping Algorithms Using Electronic Health Records.

OBJECTIVE: The aim of this study was to investigate and compare different case definitions for chronic pain to provide estimates of possible misclassification when researchers are limited by available electronic health record and administrative claims data, allowing for greater precision in case definitions. METHODS: We compared the prevalence of different case definitions for chronic pain (N = 3042) in patients with autoimmune rheumatic diseases. We estimated the prevalence of chronic pain based on 15 unique combinations of pain scores, diagnostic codes, analgesic medications, and pain interventions. RESULTS: Chronic pain prevalence was lowest in unimodal pain phenotyping algorithms: 15% using analgesic medications, 18% using pain scores, 21% using pain diagnostic codes, and 22% using pain interventions. In comparison, the prevalence using a well-validated phenotyping algorithm was 37%. The prevalence of chronic pain also increased with the increasing number (bimodal to quadrimodal) of phenotyping algorithms that comprised the multimodal phenotyping algorithms. The highest estimated chronic pain prevalence (47%) was the multimodal phenotyping algorithm that combined pain scores, diagnostic codes, analgesic medications, and pain interventions. However, this quadrimodal phenotyping algorithm yielded a 10% overestimation of chronic pain compared to the well-validated algorithm. CONCLUSION: This is the first empirical study to our knowledge that shows that established common modes of phenotyping chronic pain can lead to substantially varying estimates of the number of patients with chronic pain. These findings can be a reference for biases in case definitions for chronic pain and could be used to estimate the extent of possible misclassifications or corrections in using datasets that cannot include specific data elements.

ChatGPT Influence on Medical Decision-Making, Bias, and Equity: A Randomized Study of Clinicians Evaluating Clinical Vignettes.

In a randomized, pre-post intervention study, we evaluated the influence of a large language model (LLM) generative AI system on accuracy of physician decision-making and bias in healthcare. 50 US-licensed physicians reviewed a video clinical vignette, featuring actors representing different demographics (a White male or a Black female) with chest pain. Participants were asked to answer clinical questions around triage, risk, and treatment based on these vignettes, then asked to reconsider after receiving advice generated by ChatGPT+ (GPT4). The primary outcome was the accuracy of clinical decisions based on pre-established evidence-based guidelines. Results showed that physicians are willing to change their initial clinical impressions given AI assistance, and that this led to a significant improvement in clinical decision-making accuracy in a chest pain evaluation scenario without introducing or exacerbating existing race or gender biases. A survey of physician participants indicates that the majority expect LLM tools to play a significant role in clinical decision making.