Using natural language processing in emergency medicine health service research: A systematic review and meta-analysis.

OBJECTIVES: Natural language processing (NLP) represents one of the adjunct technologies within artificial intelligence and machine learning, creating structure out of unstructured data. This study aims to assess the performance of employing NLP to identify and categorize unstructured data within the emergency medicine (EM) setting. METHODS: We systematically searched publications related to EM research and NLP across databases including MEDLINE, Embase, Scopus, CENTRAL, and ProQuest Dissertations & Theses Global. Independent reviewers screened, reviewed, and evaluated article quality and bias. NLP usage was categorized into syndromic surveillance, radiologic interpretation, and identification of specific diseases/events/syndromes, with respective sensitivity analysis reported. Performance metrics for NLP usage were calculated and the overall area under the summary of receiver operating characteristic curve (SROC) was determined. RESULTS: A total of 27 studies underwent meta-analysis. Findings indicated an overall mean sensitivity (recall) of 82%-87%, specificity of 95%, with the area under the SROC at 0.96 (95% CI 0.94-0.98). Optimal performance using NLP was observed in radiologic interpretation, demonstrating an overall mean sensitivity of 93% and specificity of 96%. CONCLUSIONS: Our analysis revealed a generally favorable performance accuracy in using NLP within EM research, particularly in the realm of radiologic interpretation. Consequently, we advocate for the adoption of NLP-based research to augment EM health care management.

The association between electronic health information usage and patient-centered communication: a cross sectional analysis from the Health Information National Trends Survey (HINTS).

BACKGROUND: Patient-provider communication can be assessed by the patient-centered communication (PCC) score. With rapid development of electronic health (eHealth) information usage, we are uncertain of their role in PCC. Our study aims to determine the association between PCC and eHealth usage with the analysis of national representative survey data. METHODS: This is a cross sectional analysis using the Health Information National Trends Survey 5 (HINTS 5) cycle 1 to cycle 4 data (2017-2020). Seven specific questions were used for PCC assessment, and eHealth usage was divided into two types (private-eHealth and public-eHealth usage). A multivariate logistic regression was performed to determine the association between PCC and eHealth usage after the adjustment of other social, demographic, and clinical variables. RESULTS: Our study analyzed a total of 13,055 unweighted participants representing a weighted population of 791,877,728. Approximately 43% of individuals used private eHealth and 19% used public eHealth. The adjusted odds ratio (AOR) of private-eHealth usage associated with positive PCC was 1.17 (95% CI 1.02-1.35, p = 0.027). The AOR of public-eHealth usage associated with positive PCC was 0.84 (95% CI 0.71-0.99, p = 0.043). CONCLUSION: Our study found that eHealth usage association with PCC varies. Private-eHealth usage was positively associated with PCC, whereas public-eHealth usage was negatively associated with PCC.

The Association of Name Recognition, Empathy Perception, and Satisfaction With Resident Physicians' Care Amongst Patients in an Academic Emergency Department.

Background: Recognition of the provider's name, provider empathy, and the patient's satisfaction with their care are patient-provider rapport measures. This study aimed to determine: 1) resident physicians' name recognition by patients in the emergency department; and 2) name recognition in association with patient perception of the resident's empathy and their satisfaction with the resident's care. Methods: This was a prospective observational study. A patient recognizing a resident physician was defined as the patient remembering a resident's name, understanding the level of training, and understanding a resident's role in patient care. A patient's perception of resident physician empathy was measured by the Jefferson Scale of Patient Perception of Physician Empathy (JSPPPE). Patient satisfaction of the resident was measured utilizing a real-time satisfaction survey. Multivariate logistic regressions were performed to determine the association amongst patient recognition of resident physicians, JSPPPE, and patient satisfaction after adjustments were made for demographics and resident training level. Results: We enrolled 30 emergency medicine resident physicians and 191 patients. Only 26% of studied patients recognized resident physicians. High JSPPPE scores were given by 39% of patients recognizing resident physicians compared to 5% of those who were not recognized (P = 0.013). High patient satisfaction scores were recorded in 31% of patients who recognized resident physicians compared to 7% who did not (P = 0.008). The adjusted odds ratios of patient recognition of resident physicians to high JSPPPE and high satisfaction scores were 5.29 (95% confidence interval (CI): 1.33 - 21.02, P = 0.018) and 6.12 (1.84 - 20.38, P = 0.003) respectively. Conclusions: Patient recognition of resident physicians is low in our study. However, patient recognition of resident physicians is associated with a higher patient perception of physician empathy and higher patient satisfaction. Our study suggests that resident education advocating for patient recognition of their healthcare provider's status needs to be emphasized as part of patient-centered health care.

Using Machine Learning Algorithms to Predict Patient Portal Use Among Emergency Department Patients With Diabetes Mellitus.

Background: Different machine learning (ML) technologies have been applied in healthcare systems with diverse applications. We aimed to determine the model feasibility and accuracy of predicting patient portal use among diabetic patients by using six different ML algorithms. In addition, we also compared model performance accuracy with the use of only essential variables. Methods: This was a single-center retrospective observational study. From March 1, 2019 to February 28, 2020, we included all diabetic patients from the study emergency department (ED). The primary outcome was the status of patient portal use. A total of 18 variables consisting of patient sociodemographic characteristics, ED and clinic information, and patient medical conditions were included to predict patient portal use. Six ML algorithms (logistic regression, random forest (RF), deep forest, decision tree, multilayer perception, and support vector machine) were used for such predictions. During the initial step, ML predictions were performed with all variables. Then, the essential variables were chosen via feature selection. Patient portal use predictions were repeated with only essential variables. The performance accuracies (overall accuracy, sensitivity, specificity, and area under receiver operating characteristic curve (AUC)) of patient portal predictions were compared. Results: A total of 77,977 unique patients were placed in our final analysis. Among them, 23.4% (18,223) patients were diabetic mellitus (DM). Patient portal use was found in 26.9% of DM patients. Overall, the accuracy of predicting patient portal use was above 80% among five out of six ML algorithms. The RF outperformed the others when all variables were used for patient portal predictions (accuracy 0.9876, sensitivity 0.9454, specificity 0.9969, and AUC 0.9712). When only eight essential variables were chosen, RF still outperformed the others (accuracy 0.9876, sensitivity 0.9374, specificity 0.9932, and AUC 0.9769). Conclusion: It is possible to predict patient portal use outcomes when different ML algorithms are used with fair performance accuracy. However, with similar prediction accuracies, the use of feature selection techniques can improve the interpretability of the model by addressing the most relevant features.

Dedicated emergency department ultrasound rotation improves residents' ultrasound knowledge and interpretation skills.

BACKGROUND: Graduates of Emergency Medicine (EM) residency training programs are expected to be proficient in ultrasound. However, best practices for teaching residents ultrasonography has yet to be determined. STUDY OBJECTIVES: To determine if a dedicated Emergency Department (ED) ultrasound rotation objectively improves residents' EM ultrasound knowledge, interpretation accuracy, and clinical decision-making based on ultrasound findings. METHODS: EM residents completing a required ED-based ultrasound rotation were prospectively studied. Before the start of the rotation, each resident completed a 20-question pre-test. At the end of the rotation, residents completed a 20-question post-test. Both tests covered physics, trauma (focused assessment with sonography for trauma), first-trimester pregnancy, aorta, biliary, echocardiography, and vascular sonography, using a multiple-choice format. In both tests, ultrasound images were included in 11 of the 20 questions. The questions were divided into three categories: knowledge-based (8 questions), interpretation (9 questions), and clinical decision-making (3 questions), for both tests. Scores on pre-tests and post-tests were compared using a Wilcoxon signed-rank test. RESULTS: During the 2-year study period, 21 residents completed the rotation. The median pre-test score was 16 (interquartile range [IQR] 14.5-17), compared to a median post-test score of 19 (IQR 18-20), p < 0.001. CONCLUSIONS: A dedicated ED ultrasound rotation improves residents' EM ultrasound knowledge and interpretation accuracy based on ultrasound findings, as measured by improvement on ultrasound test scores.

The impact of emergency department overcrowding on resident education.

BACKGROUND: Few studies have evaluated the effect of Emergency Department (ED) overcrowding on resident education. OBJECTIVES: To determine the impact of ED overcrowding on Emergency Medicine (EM) resident education. MATERIALS AND METHODS: A prospective cross-sectional study was performed from March to May 2009. Second- and third-year EM residents, blinded to the research objective, completed a questionnaire at the end of each shift. Residents were asked to evaluate the educational quality of each shift using a 10-point Likert scale. Number of patients seen and procedures completed were recorded. Responses were divided into ED overcrowding (group O) and non-ED overcrowding (group N) groups. ED overcrowding was defined as >2 h of ambulance diversion per shift. Questionnaire responses were compared using Mann-Whitney U tests. Number of patients and procedures were compared using unpaired T-tests. RESULTS: During the study period, 125 questionnaires were completed; 54 in group O and 71 in group N. For group O, the median educational value score was 8 (interquartile range [IQR] 7-10), compared to 8 (IQR 8-10) for group N (p = 0.24). Mean number of patients seen in group O was 12.3 (95% confidence interval [CI] 11.4-13.2), compared to 13.9 (95% CI 12.7-15) in group N (p = 0.034). In group O, mean number of procedures was 0.9 (95% CI 0.6-1.2), compared to 1.3 (95% CI 1-1.6) in group N (p = 0.047). CONCLUSIONS: During overcrowding, EM residents saw fewer patients and performed fewer procedures. However, there was no significant difference in resident perception of educational value during times of overcrowding vs. non-overcrowding.

Assessing interpersonal and communication skills in emergency medicine.

Interpersonal and communication skills (ICS) are a key component of several competency-based schemata and key competency in the set of six Accreditation Council for Graduate Medical Education (ACGME) core competencies. With the shift toward a competency-based educational framework, the importance of robust learner assessment becomes paramount. The journal Academic Emergency Medicine (AEM) hosted a consensus conference to discuss education research in emergency medicine (EM). This article summarizes the initial preparatory research that was conducted to brief consensus conference attendees and reports the results of the consensus conference breakout session as it pertains to ICS assessment of learners. The goals of this consensus conference session were to twofold: 1) to determine the state of assessment of observable learner performance and 2) to determine a research agenda within the ICS field for medical educators. The working group identified six key recommendations for medical educators and researchers.

Techniques for teaching electrocardiogram interpretation: self-directed learning is less effective than a workshop or lecture.

CONTEXT: Teaching 12-lead electrocardiogram (ECG) interpretation to students and residents is a challenge for medical educators. To date, few studies have compared the effectiveness of different techniques used for ECG teaching. OBJECTIVES: This study aimed to determine if common teaching techniques, such as those involving workshops, lectures and self-directed learning (SDL), increase medical students' ability to correctly interpret ECGs. It also aimed to compare the effectiveness of these formats. METHODS: This was a prospective randomised study conducted over a 28-month period. Year 4 medical students were randomised to receive teaching in ECG interpretation using one of three teaching formats: workshop, lecture or SDL. All three formats covered the same content. Students were administered three tests: a pre-test (before teaching); a post-test (immediately after teaching), and a retention test (1 week after teaching). Each tested the same content using 25 questions worth 1 point each. A mixed-model repeated-measures analysis of variance (anova) with least squares post hoc analysis was conducted to determine if differences in test scores between the formats were statistically significant. RESULTS: Of the 223 students for whom data were analysed, 79 were randomised to a workshop, 82 to a lecture-based format and 62 to SDL. All three teaching formats resulted in a statistically significant improvement in individual test scores (p < 0.001). Comparison of the lecture- and workshop-based formats demonstrated no difference in test scores (marginal mean [MM] for both formats = 12.4, 95% confidence interval [95% CI] 11.7-13.2]; p = 0.99). Test scores of students using SDL (MM = 10.7, 95% CI 9.8-11.5) were lower than those of students in the workshop (p = 0.003) and lecture (p = 0.002) groups. CONCLUSIONS: Compared with those taught using workshop- and lecture-based formats, medical students learning ECG interpretation by SDL had lower test scores.

Use of a sedation-analgesia datasheet in closed shoulder reductions.

A three-page conscious sedation (CS) monitoring datasheet and sedation-analgesia policy were implemented at the Lincoln Hospital Emergency Department (LHED) to maintain compliance with JCAHO and New York State standards on CS monitoring. The datasheet included areas for physical examination, medications, and vital signs. To determine effects of the new datasheet and policy, charts containing all closed shoulder reductions done at LHED from April 3, 1996 to June 30, 1999 (n = 237) were reviewed for use of CS, defined as the use of an analgesic and sedative-hypnotic concurrently. Utilization of CS during equal time periods of 591 days before and after datasheet implementation was analyzed and compared. Before use of the datasheet, 64.3% received CS before closed shoulder reduction, compared with 41.8% after its use began (p < 0.05). A subsequent survey was conducted to determine physician perception of the CS datasheet. Attending-level LHED physicians suggested that use of the datasheet increased charting time and liability.