Measuring the Impact of AI in the Diagnosis of Hospitalized Patients: A Randomized Clinical Vignette Survey Study.

Importance: Artificial intelligence (AI) could support clinicians when diagnosing hospitalized patients; however, systematic bias in AI models could worsen clinician diagnostic accuracy. Recent regulatory guidance has called for AI models to include explanations to mitigate errors made by models, but the effectiveness of this strategy has not been established. Objectives: To evaluate the impact of systematically biased AI on clinician diagnostic accuracy and to determine if image-based AI model explanations can mitigate model errors. Design, Setting, and Participants: Randomized clinical vignette survey study administered between April 2022 and January 2023 across 13 US states involving hospitalist physicians, nurse practitioners, and physician assistants. Interventions: Clinicians were shown 9 clinical vignettes of patients hospitalized with acute respiratory failure, including their presenting symptoms, physical examination, laboratory results, and chest radiographs. Clinicians were then asked to determine the likelihood of pneumonia, heart failure, or chronic obstructive pulmonary disease as the underlying cause(s) of each patient's acute respiratory failure. To establish baseline diagnostic accuracy, clinicians were shown 2 vignettes without AI model input. Clinicians were then randomized to see 6 vignettes with AI model input with or without AI model explanations. Among these 6 vignettes, 3 vignettes included standard-model predictions, and 3 vignettes included systematically biased model predictions. Main Outcomes and Measures: Clinician diagnostic accuracy for pneumonia, heart failure, and chronic obstructive pulmonary disease. Results: Median participant age was 34 years (IQR, 31-39) and 241 (57.7%) were female. Four hundred fifty-seven clinicians were randomized and completed at least 1 vignette, with 231 randomized to AI model predictions without explanations, and 226 randomized to AI model predictions with explanations. Clinicians' baseline diagnostic accuracy was 73.0% (95% CI, 68.3% to 77.8%) for the 3 diagnoses. When shown a standard AI model without explanations, clinician accuracy increased over baseline by 2.9 percentage points (95% CI, 0.5 to 5.2) and by 4.4 percentage points (95% CI, 2.0 to 6.9) when clinicians were also shown AI model explanations. Systematically biased AI model predictions decreased clinician accuracy by 11.3 percentage points (95% CI, 7.2 to 15.5) compared with baseline and providing biased AI model predictions with explanations decreased clinician accuracy by 9.1 percentage points (95% CI, 4.9 to 13.2) compared with baseline, representing a nonsignificant improvement of 2.3 percentage points (95% CI, -2.7 to 7.2) compared with the systematically biased AI model. Conclusions and Relevance: Although standard AI models improve diagnostic accuracy, systematically biased AI models reduced diagnostic accuracy, and commonly used image-based AI model explanations did not mitigate this harmful effect. Trial Registration: ClinicalTrials.gov Identifier: NCT06098950.

Camphor Oil Toxicity: A Case Report.

Camphor is a highly toxic ingredient that can be found in commonly used rubs and preparations such as Tiger balm and Vicks. There is a wide range of symptoms resulting from camphor oil toxicity, manifesting in sweating and agitation and can progress to more serious symptoms of seizures, cardiac arrhythmias, and cardiopulmonary arrest. We present a 61-year-old male, who is a known case of major depressive disorder, was brought to the emergency department on 10/09/2022, two hours after ingesting approximately 500 mL of camphor oil in its liquid form. He developed two episodes of tonic-clonic seizures at home and then later had another episode in the emergency department. As he presented to the emergency room, he was confused, agitated, restless, and diaphoretic. The management in the Emergency Department started with assessing his airway and administration of intravenous (IV) benzodiazepines and IV fluids. The ECG revealed sinus rhythm with borderline QT and QRS. During his stay in the emergency room, his mental status worsened and he became more confused and restless, and he developed another tonic-conic seizure. Therefore, he was intubated. The patient was shifted and managed in the intensive care unit, and 48 hours later the patient was extubated. This case report illustrates the importance of addressing the potential risks of home remedies as they are increasingly being used by the population considering them as safe. Camphor, being the most cultivated essential oil, is a highly toxic compound that, even in very small concentrations, can be lethal to infants and children. It is a component of numerous over-the-counter remedies and has the potential for accidental consumption. Generalized tonic-clonic seizure being the most prominent manifestation which can occur as early as five minutes after exposure needs to be anticipated and treated accordingly. Treatment for symptomatic patients is primarily supportive with special attention paid to QRS complex widening in the ECG.

Combining chest X-rays and electronic health record (EHR) data using machine learning to diagnose acute respiratory failure.

OBJECTIVE: When patients develop acute respiratory failure (ARF), accurately identifying the underlying etiology is essential for determining the best treatment. However, differentiating between common medical diagnoses can be challenging in clinical practice. Machine learning models could improve medical diagnosis by aiding in the diagnostic evaluation of these patients. MATERIALS AND METHODS: Machine learning models were trained to predict the common causes of ARF (pneumonia, heart failure, and/or chronic obstructive pulmonary disease [COPD]). Models were trained using chest radiographs and clinical data from the electronic health record (EHR) and applied to an internal and external cohort. RESULTS: The internal cohort of 1618 patients included 508 (31%) with pneumonia, 363 (22%) with heart failure, and 137 (8%) with COPD based on physician chart review. A model combining chest radiographs and EHR data outperformed models based on each modality alone. Models had similar or better performance compared to a randomly selected physician reviewer. For pneumonia, the combined model area under the receiver operating characteristic curve (AUROC) was 0.79 (0.77-0.79), image model AUROC was 0.74 (0.72-0.75), and EHR model AUROC was 0.74 (0.70-0.76). For heart failure, combined: 0.83 (0.77-0.84), image: 0.80 (0.71-0.81), and EHR: 0.79 (0.75-0.82). For COPD, combined: AUROC = 0.88 (0.83-0.91), image: 0.83 (0.77-0.89), and EHR: 0.80 (0.76-0.84). In the external cohort, performance was consistent for heart failure and increased for COPD, but declined slightly for pneumonia. CONCLUSIONS: Machine learning models combining chest radiographs and EHR data can accurately differentiate between common causes of ARF. Further work is needed to determine how these models could act as a diagnostic aid to clinicians in clinical settings.

Augmenting existing deterioration indices with chest radiographs to predict clinical deterioration.

IMPORTANCE: When hospitals are at capacity, accurate deterioration indices could help identify low-risk patients as potential candidates for home care programs and alleviate hospital strain. To date, many existing deterioration indices are based entirely on structured data from the electronic health record (EHR) and ignore potentially useful information from other sources. OBJECTIVE: To improve the accuracy of existing deterioration indices by incorporating unstructured imaging data from chest radiographs. DESIGN, SETTING, AND PARTICIPANTS: Machine learning models were trained to predict deterioration of patients hospitalized with acute dyspnea using existing deterioration index scores and chest radiographs. Models were trained on hospitalized patients without coronavirus disease 2019 (COVID-19) and then subsequently tested on patients with COVID-19 between January 2020 and December 2020 at a single tertiary care center who had at least one radiograph taken within 48 hours of hospital admission. MAIN OUTCOMES AND MEASURES: Patient deterioration was defined as the need for invasive or non-invasive mechanical ventilation, heated high flow nasal cannula, IV vasopressor administration or in-hospital mortality at any time following admission. The EPIC deterioration index was augmented with unstructured data from chest radiographs to predict risk of deterioration. We compared discriminative performance of the models with and without incorporating chest radiographs using area under the receiver operating curve (AUROC), focusing on comparing the fraction and total patients identified as low risk at different negative predictive values (NPV). RESULTS: Data from 6278 hospitalizations were analyzed, including 5562 hospitalizations without COVID-19 (training cohort) and 716 with COVID-19 (216 in validation, 500 in held-out test cohort). At a NPV of 0.95, the best-performing image-augmented deterioration index identified 49 more (9.8%) individuals as low-risk compared to the deterioration index based on clinical data alone in the first 48 hours of admission. At a NPV of 0.9, the EPIC image-augmented deterioration index identified 26 more individuals (5.2%) as low-risk compared to the deterioration index based on clinical data alone in the first 48 hours of admission. CONCLUSION AND RELEVANCE: Augmenting existing deterioration indices with chest radiographs results in better identification of low-risk patients. The model augmentation strategy could be used in the future to incorporate other forms of unstructured data into existing disease models.

Learning Concept Credible Models for Mitigating Shortcuts.

During training, models can exploit spurious correlations as shortcuts, resulting in poor generalization performance when shortcuts do not persist. In this work, assuming access to a representation based on domain knowledge (i.e., known concepts) that is invariant to shortcuts, we aim to learn robust and accurate models from biased training data. In contrast to previous work, we do not rely solely on known concepts, but allow the model to also learn unknown concepts. We propose two approaches for mitigating shortcuts that incorporate domain knowledge, while accounting for potentially important yet unknown concepts. The first approach is two-staged. After fitting a model using known concepts, it accounts for the residual using unknown concepts. While flexible, we show that this approach is vulnerable when shortcuts are correlated with the unknown concepts. This limitation is addressed by our second approach that extends a recently proposed regularization penalty. Applied to two real-world datasets, we demonstrate that both approaches can successfully mitigate shortcut learning.

Respecting Autonomy And Enabling Diversity: The Effect Of Eligibility And Enrollment On Research Data Demographics.

Many promising advances in precision health and other Big Data research rely on large data sets to analyze correlations among genetic variants, behavior, environment, and outcomes to improve population health. But these data sets are generally populated with demographically homogeneous cohorts. We conducted a retrospective cohort study of patients at a major academic medical center during 2012-19 to explore how recruitment and enrollment approaches affected the demographic diversity of participants in its research biospecimen and data bank. We found that compared with the overall clinical population, patients who consented to enroll in the research data bank were significantly less diverse in terms of age, sex, race, ethnicity, and socioeconomic status. Compared with patients who were recruited for the data bank, patients who enrolled were younger and less likely to be Black or African American, Asian, or Hispanic. The overall demographic diversity of the data bank was affected as much (and in some cases more) by which patients were considered eligible for recruitment as by which patients consented to enroll. Our work underscores the need for systemic commitment to diversify data banks so that different communities can benefit from research.

ACUTE ABDOMEN systemic sonographic approach to acute abdomen in emergency department: a case series.

BACKGROUND: Acute abdomen is a medical emergency with a wide spectrum of etiologies. Point-of-care ultrasound (POCUS) can help in early identification and management of the causes. The ACUTE-ABDOMEN protocol was created by the authors to aid in the evaluation of acute abdominal pain using a systematic sonographic approach, integrating the same core ultrasound techniques already in use-into one mnemonic. This mnemonic ACUTE means: A: abdominal aortic aneurysm; C: collapsed inferior vena cava; U: ulcer (perforated viscus); T: trauma (free fluid); E: ectopic pregnancy, followed by ABDOMEN which stands: A: appendicitis; B: biliary tract; D: distended bowel loop; O: obstructive uropathy; Men: testicular torsion/Women: ovarian torsion. The article discusses two cases of abdominal pain the diagnosis and management of which were directed and expedited as a result of using the ACUTE-ABDOMEN protocol. The first case was of a 33-year-old male, who presented with a 3-day history of abdominal pain, vomiting and constipation. Physical exam revealed a soft abdomen with generalized tenderness and normal bowel sounds. Laboratory tests were normal. A bedside ultrasound done using the ACUTE-ABDOMEN protocol showed signs of intussusception. This was confirmed by CT-abdomen. The second case was of a 70-year-old female, a known case of diabetes and hypertension, who presented with a 3-hour history of abdominal pain, vomiting and diarrhea. She had a normal physical exam and laboratory studies. Her symptoms mimicking simple gastroenteritis had improved. However, bedside ultrasound, using the ACUTE-ABDOMEN protocol showed localized free fluid with dilated small bowel loop in right lower quadrant with absent peristalsis. A CT abdomen confirmed a diagnosis of intestinal obstruction. These two cases demonstrate that the usefulness of applying POCUS in a systematic method-like the "ACUTE-ABDOMEN" approach-can aid in patient diagnosis and management. CASE PRESENTATION: We are presenting two cases of undifferentiated acute abdomen pain, where ACUTE ABDOMEN sonographic approach was applied and facilitated the accurate patient management and disposition. CONCLUSION: ACUTE ABDOMEN sonographic approach in acute abdomen can play an important role in ruling out critical diagnosis, and can guide emergency physician or any critical care physician in patient management.