Away Rotation Applications in Emergency Medicine: Medical Student Behaviors, Outcomes, and Stressors.

BACKGROUND: Completing an emergency medicine (EM) away rotation is integral to matching successfully into an EM residency program. The demand for EM away rotations (ARs) drives students to submit numerous applications without evidence-based recommendations to guide stakeholders on the approach or number to submit. OBJECTIVES: We conducted a survey study of EM-bound fourth-year medical students to gain insight into their AR application experiences, outcomes, and perceptions. METHODS: We distributed a 40-item questionnaire to EM applicants in Fall 2018 via e-mail through the Clerkship Directors in Emergency Medicine, Council of Residency Directors in EM, and Emergency Medicine Residents' Association listservs. Responses were evaluated using quantitative and qualitative analysis. Primary outcomes were the number of AR applications submitted and AR offers received by students. Secondary outcomes were students' self-assessment of their competitiveness, differences in AR application numbers by degree type, sources of student advising, and student perceptions of the AR application process. RESULTS: There were 253 respondents, consisting of 192 allopathic (MD) and 61 osteopathic (DO) medical students, who met the inclusion criteria, representing about 10% of the applicant pool. On average, students submitted 13.97 applications (95% confidence interval [CI] 11.59-16.35), received 3.25 offers (95% CI 3.01-3.49), and accepted 2.22 offers (95% CI 2.08-2.36). DO candidates submitted twice as many applications as MD candidates while experiencing a similar rate of offers received. Peer influence (n = 154, 61%), peer online advising networks (n = 83, 33%), and self-assessment (n = 114, 45%) were the most often reported causes of increased applications; cost (n = 104, 41%) and geographic limitations (n = 114, 45%) were the most often reported causes of decreased applications. Open-response analysis revealed frustration with lack of standardization (n = 44, 29.5%), insufficient transparency on available positions (n = 37, 24.8%), limited communication (n = 30, 20.1%), and cost (n = 12, 8.1%). CONCLUSIONS: This study showed that, as a whole, students received one away rotation offer for every four to five applications submitted. It clarified factors contributing to increased EM away rotation application submissions and associated stressors inherent in the application experience. Our findings offer insights to inform advising recommendations. They also suggest that stakeholders consider standardizing the process and improve communication over spot availability and application status.

A Model Partnership: Mentoring Underrepresented Students in Medicine (URiM) in Emergency Medicine.

INTRODUCTION: Creating a racially and ethnically diverse workforce remains a challenge for medical specialties, including emergency medicine (EM). One area to examine is a partnership between a predominantly white institution (PWI) with a historically black college and university (HBCU) to determine whether this partnership would increase the number of underrepresented in medicine (URiM) in EM who are from a HBCU. METHODS: Twenty years ago Emory Department of Emergency Medicine began its collaboration with Morehouse School of Medicine (MSM) to provide guidance to MSM students who were interested in EM. Since its inception, our engagement and intervention has evolved over time to include mentorship and guidance from the EM clerkship director, program director, and key faculty. RESULTS: Since the beginning of the MSM-Emory EM partnership, 115 MSM students have completed an EM clerkship at Emory. Seventy-two of those students (62.6%) have successfully matched into an EM residency program. Of those who matched into EM, 22 (32%) have joined the Emory EM residency program with the remaining 50 students matching at 40 other EM programs across the nation. CONCLUSION: Based on our experience and outcomes with the Emory-MSM partnership, we are confident that a partnership with an HBCU school without an EM residency should be considered by residency programs to increase the number of URiM students in EM, which could perhaps translate to other specialties.

Gender Bias in Simulation-Based Assessments of Emergency Medicine Residents.

BACKGROUND: Gender-related disparities persist in medicine and medical education. Prior work has found differences in medical education assessments based on gender. OBJECTIVE: We hypothesized that gender bias would be mitigated in a simulation-based assessment. METHODS: We conducted a retrospective cohort study of emergency medicine residents at a single, urban residency program. Beginning in spring 2013, residents participated in mandatory individual simulation assessments. Twelve simulated cases were included in this study. Rating forms mapped milestone language to specific observable behaviors. A Bayesian regression was used to evaluate the effect of resident and rater gender on assessment scores. Both 95% credible intervals (CrIs) and a Region of Practical Equivalence approach were used to evaluate the results. RESULTS: Participants included 48 faculty raters (25 men [52%]) and 102 residents (47 men [46%]). The difference in scores between male and female residents (M = -0.58, 95% CrI -3.31-2.11), and male and female raters (M = 2.87, 95% CrI -0.43-6.30) was small and 95% CrIs overlapped with 0. The 95% CrI for the interaction between resident and rater gender also overlapped with 0 (M = 0.41, 95% CrI -3.71-4.23). CONCLUSIONS: In a scripted and controlled system of assessments, there were no differences in scores due to resident or rater gender.

Medical Student Documentation in the Electronic Medical Record: Patterns of Use and Barriers.

INTRODUCTION: Electronic health records (EHR) have become ubiquitous in emergency departments. Medical students rotating on emergency medicine (EM) clerkships at these sites have constant exposure to EHRs as they learn essential skills. The Association of American Medical Colleges (AAMC), the Liaison Committee on Medical Education (LCME), and the Alliance for Clinical Education (ACE) have determined that documentation of the patient encounter in the medical record is an essential skill that all medical students must learn. However, little is known about the current practices or perceived barriers to student documentation in EHRs on EM clerkships. METHODS: We performed a cross-sectional study of EM clerkship directors at United States medical schools between March and May 2016. A 13-question IRB-approved electronic survey on student documentation was sent to all EM clerkship directors. Only one response from each institution was permitted. RESULTS: We received survey responses from 100 institutions, yielding a response rate of 86%. Currently, 63% of EM clerkships allow medical students to document a patient encounter in the EHR. The most common reasons cited for not permitting students to document a patient encounter were hospital or medical school rule forbidding student documentation (80%), concern for medical liability (60%), and inability of student notes to support medical billing (53%). Almost 95% of respondents provided feedback on student documentation with supervising faculty being the most common group to deliver feedback (92%), followed by residents (64%). CONCLUSION: Close to two-thirds of medical students are allowed to document in the EHR on EM clerkships. While this number is robust, many organizations such as the AAMC and ACE have issued statements and guidelines that would look to increase this number even further to ensure that students are prepared for residency as well as their future careers. Almost all EM clerkships provided feedback on student documentation indicating the importance for students to learn this skill.

Scribe Impacts on Provider Experience, Operations, and Teaching in an Academic Emergency Medicine Practice.

INTRODUCTION: Physicians dedicate substantial time to documentation. Scribes are sometimes used to improve efficiency by performing documentation tasks, although their impacts have not been prospectively evaluated. Our objective was to assess a scribe program's impact on emergency department (ED) throughput, physician time utilization, and job satisfaction in a large academic emergency medicine practice. METHODS: We evaluated the intervention using pre- and post-intervention surveys and administrative data. All site physicians were included. Pre- and post-intervention data were collected in four-month periods one year apart. Primary outcomes included changes in monthly average ED length of stay (LOS), provider-specific average relative value units (RVUs) per hour (raw and normalized to volume), self-reported estimates of time spent teaching, self-reported estimates of time spent documenting, and job satisfaction. We analyzed data using descriptive statistics and appropriate tests for paired pre-post differences in continuous, categorical, and ranked variables. RESULTS: Pre- and post-survey response rates were 76.1% and 69.0%, respectively. Most responded positively to the intervention, although 9.5% reported negative impressions. There was a 36% reduction (25%-50%; p<0.01) in time spent documenting and a 30% increase (11%-46%, p<0.01) in time spent in direct patient contact. No statistically significant changes were seen in job satisfaction or perception of time spent teaching. ED volume increased by 88 patients per day (32-146, p=0.04) pre- to post- and LOS was unchanged; rates of patients leaving against medical advice dropped, and rates of patients leaving without being seen increased. RVUs per hour increased 5.5% and per patient 5.3%; both were statistically significant. No statistically significant changes were seen in patients seen per hour. There was moderate correlation between changes in ED volume and changes in productivity metrics. CONCLUSION: Scribes were well received in our practice. Documentation time was substantially reduced and redirected primarily to patient care. Despite an ED volume increase, LOS was maintained, with fewer patients leaving against medical advice but more leaving without being seen. RVUs per hour and per patient both increased.

Objective structured clinical examinations provide valid clinical skills assessment in emergency medicine education.

INTRODUCTION: Evaluation of emergency medicine (EM) learners based on observed performance in the emergency department (ED) is limited by factors such as reproducibility and patient safety. EM educators depend on standardized and reproducible assessments such as the objective structured clinical examination (OSCE). The validity of the OSCE as an evaluation tool in EM education has not been previously studied. The objective was to assess the validity of a novel management-focused OSCE as an evaluation instrument in EM education through demonstration of performance correlation with established assessment methods and case item analysis. METHODS: We conducted a prospective cohort study of fourth-year medical students enrolled in a required EM clerkship. Students enrolled in the clerkship completed a five-station EM OSCE. We used Pearson's coefficient to correlate OSCE performance with performance in the ED based on completed faculty evaluations. Indices of difficulty and discrimination were computed for each scoring item. RESULTS: We found a moderate and statistically-significant correlation between OSCE score and ED performance score [r(239) =0.40, p<0.001]. Of the 34 OSCE testing items the mean index of difficulty was 63.0 (SD =23.0) and the mean index of discrimination was 0.52 (SD =0.21). CONCLUSION: Student performance on the OSCE correlated with their observed performance in the ED, and indices of difficulty and differentiation demonstrated alignment with published best-practice testing standards. This evidence, along with other attributes of the OSCE, attest to its validity. Our OSCE can be further improved by modifying testing items that performed poorly and by examining and maximizing the inter-rater reliability of our evaluation instrument.

Alliance for clinical education perspective paper: recommendations for redesigning the "final year" of medical school.

BACKGROUND: Although medical school typically lasts 4 years, little attention has been devoted to the structure of the educational experience that takes place during the final year of medical school. SUMMARY: In this perspectives paper, we outline goals for the 4th year of medical school to facilitate a transition from undergraduate to graduate medical education. We provide recommendations for capstone courses, subinternship rotations, and specialty-specific schedules, and we conclude with recommendations to medical students and medical schools for how to use the recommendations contained in this document. CONCLUSIONS: We provide an overview of general competencies and specialty specific recommendations to serve as a foundation for medical schools to develop robust 4th-year curricula and for medical students to plan their 4th-year schedules.

Assessing patient care: summary of the breakout group on assessment of observable learner performance.

There is an established expectation that physicians in training demonstrate competence in all aspects of clinical care prior to entering professional practice. Multiple methods have been used to assess competence in patient care, including direct observation, simulation-based assessments, objective structured clinical examinations (OSCEs), global faculty evaluations, 360-degree evaluations, portfolios, self-reflection, clinical performance metrics, and procedure logs. A thorough assessment of competence in patient care requires a mixture of methods, taking into account each method's costs, benefits, and current level of evidence. At the 2012 Academic Emergency Medicine (AEM) consensus conference on educational research, one breakout group reviewed and discussed the evidence supporting various methods of assessing patient care and defined a research agenda for the continued development of specific assessment methods based on current best practices. In this article, the authors review each method's supporting reliability and validity evidence and make specific recommendations for future educational research.

Reporter-Interpreter-Manager-Educator (RIME) descriptive ratings as an evaluation tool in an emergency medicine clerkship.

BACKGROUND: Emergency Medicine (EM) clerkships traditionally assess students using numerical ratings of clinical performance. The descriptive ratings of the Reporter, Interpreter, Manager, and Educator (RIME) method have been shown to be valuable in other specialties. OBJECTIVES: We hypothesized that the RIME descriptive ratings would correlate with clinical performance and examination scores in an EM clerkship, indicating that the RIME ratings are a valid measure of performance. METHODS: This was a prospective cohort study of an evaluation instrument for 4(th)-year medical students completing an EM rotation. This study received exempt Institutional Review Board status. EM faculty and residents completed shift evaluation forms including both numerical and RIME ratings. Students completed a final examination. Mean scores for RIME and clinical evaluations were calculated. Linear regression models were used to determine whether RIME ratings predicted clinical evaluation scores or final examination scores. RESULTS: Four hundred thirty-nine students who completed the EM clerkship were enrolled in the study. After excluding items with missing data, there were 2086 evaluation forms (based on 289 students) available for analysis. There was a clear positive relationship between RIME category and clinical evaluation score (r(2)=0.40, p<0.01). RIME ratings correlated most strongly with patient management skills and least strongly with humanistic qualities. A very weak correlation was seen with RIME and final examination. CONCLUSION: We found a positive association between RIME and clinical evaluation scores, suggesting that RIME is a valid clinical evaluation instrument. RIME descriptive ratings can be incorporated into EM evaluation instruments and provides useful data related to patient management skills.

A core competency-based objective structured clinical examination (OSCE) can predict future resident performance.

OBJECTIVES: This study evaluated the ability of an objective structured clinical examination (OSCE) administered in the first month of residency to predict future resident performance in the Accreditation Council for Graduate Medical Education (ACGME) core competencies. METHODS: Eighteen Postgraduate Year 1 (PGY-1) residents completed a five-station OSCE in the first month of postgraduate training. Performance was graded in each of the ACGME core competencies. At the end of 18 months of training, faculty evaluations of resident performance in the emergency department (ED) were used to calculate a cumulative clinical evaluation score for each core competency. The correlations between OSCE scores and clinical evaluation scores at 18 months were assessed on an overall level and in each core competency. RESULTS: There was a statistically significant correlation between overall OSCE scores and overall clinical evaluation scores (R = 0.48, p < 0.05) and in the individual competencies of patient care (R = 0.49, p < 0.05), medical knowledge (R = 0.59, p < 0.05), and practice-based learning (R = 0.49, p < 0.05). No correlation was noted in the systems-based practice, interpersonal and communication skills, or professionalism competencies. CONCLUSIONS: An early-residency OSCE has the ability to predict future postgraduate performance on a global level and in specific core competencies. Used appropriately, such information can be a valuable tool for program directors in monitoring residents' progress and providing more tailored guidance.