The Performance of ChatGPT-4 and Gemini Ultra 1.0 for Quality Assurance Review in Emergency Medical Services Chest Pain Calls.

OBJECTIVES: This study assesses the feasibility, inter-rater reliability, and accuracy of using OpenAI's ChatGPT-4 and Google's Gemini Ultra large language models (LLMs), for Emergency Medical Services (EMS) quality assurance. The implementation of these LLMs for EMS quality assurance has the potential to significantly reduce the workload on medical directors and quality assurance staff by automating aspects of the processing and review of patient care reports. This offers the potential for more efficient and accurate and identification of areas requiring improvement, thereby potentially enhancing patient care outcomesMETHODS: Two expert human reviewers, ChatGPT GPT-4, and Gemini Ultra assessed and rated 150 consecutively sampled and anonymized prehospital records from 2 large urban EMS agencies for adherence to 2020 National Association of State EMS metrics for cardiac care. We evaluated the accuracy of scoring, inter-rater reliability, and review efficiency. The inter-rater reliability for the dichotomous outcome of each EMS metric was measured using the kappa statistic.RESULTS: Human reviewers showed high interrater reliability, with 91.2% agreement and a kappa coefficient, 0.782 (0.654-0.910). ChatGPT-4 achieved substantial agreement with human reviewers in EKG documentation and aspirin administration (76.2% agreement, kappa coefficient, 0.401 (0.334-0.468), but performance varied across other metrics. Gemini Ultra's evaluation was discontinued due to poor performance. No significant differences were observed in median review times: 01:28 minutes (IQR 1:12 - 1:51 min) per human chart review, 01:24 minutes (IQR 01:09 - 01:53 min) per ChatGPT-4 chart review (p = 0.46), and 01:50 minutes (IQR 01:10-03:34 min) per Gemini Ultra review (p = 0.06).CONCLUSIONS: Large language models demonstrate potential in supporting quality assurance by effectively and objectively extracting data elements. However, their accuracy in interpreting non-standardized and time-sensitive details remains inferior to human evaluators. Our findings suggest that current LLMs may best offer supplemental support to the human review processes, but their value remains limited. Enhancements in LLM training and integration are recommended for improved and more reliable performance in the quality assurance processes.

The impact of time to defibrillation on return of spontaneous circulation in out-of-hospital cardiac arrest patients with recurrent shockable rhythms.

OBJECTIVE: Optimal timing for subsequent defibrillation attempts for Out-of-hospital cardiac arrest (OHCA) patients with recurrent VF/pVT is uncertain. We investigated the relationship between VF/pVT duration and return of spontaneous circulation (ROSC) in OHCA patients with recurrent shockable rhythms. METHODS: We analyzed data from the Salt Lake City Fire Department (SLCFD) spanning from 2012 to 2023. The implementation of rhythm-filtering technology since 2011 enabled real-time rhythm interpretation during CPR, with local protocols allowing early defibrillation for recurrent/refractory VF/pVT cases. We included patients experiencing four or five episodes of VF and pVT rhythms and employed generalized estimating equation (GEE) regression analysis to examine the association between VF/pVT durations preceding recurrent defibrillation and return of spontaneous circulation (ROSC). RESULTS: Analysis of 622 appropriate shocks showed that patients achieving ROSC had significantly shorter median VF/pVT duration than those who did not achieve ROSC (0.83 minutes vs. 1.2 minutes, p = 0.004). Adjusted analysis of those with 4 VF/pVT episodes (N = 142) revealed that longer VF/pVT durations were associated with lower odds of achieving ROSC (odds ratio: 0.81, 95% CI: 0.72-0.93, p = 0.005). Every one-minute delay in intra-arrest defibrillation is predicted to decrease the likelihood of achieving ROSC by 19%. CONCLUSION: Every one-minute increase in intra-arrest VF/pVT duration was associated with a statistically significant 19% decrease in the chance of achieving ROSC. This highlights the importance of reducing time to shock in managing recurrent VF/pVT. The findings suggest reevaluating the current recommendations of two minutes intervals for rhythm check and shock delivery.

Buprenorphine for Opioid Use Disorder in the Emergency Department: A Retrospective Chart Review.

INTRODUCTION: Emergency care providers routinely treat patients with acute presentations and sequelae of opioid use disorder. An emergency physician and pharmacist implemented a protocol using buprenorphine for the treatment of patients with opioid withdrawal at an academic, Level I trauma center. We describe our experience regarding buprenorphine implementation in the emergency department (ED), characteristics of patients who received buprenorphine, and rates of outpatient follow-up. METHODS: We conducted a retrospective chart review of all patients in the ED for whom buprenorphine was administered to treat opioid withdrawal during an 18-month period from January 30, 2017-July 31, 2018. Data extraction of a priori-defined variables was recorded. We used descriptive statistics to characterize the cohort of patients. RESULTS: A total of 77 patients were included for analysis. Thirty-three patients (43%) who received buprenorphine did not present with the chief complaint of opioid withdrawal. Most patients (74%) who received buprenorphine last used heroin, and presented in moderate opioid withdrawal. One case of precipitated withdrawal occurred after buprenorphine administration. Twenty-three (30%) patients received outpatient follow-up. CONCLUSIONS: This study underscores the safety of ED-initiated buprenorphine and that buprenorphine administration in the ED is feasible and effective.