Identifying the prevalence and characteristics of diversity, equity, and inclusion leaders in academic emergency medicine.

Objectives: Our study aims to better understand and describe the current state of diversity, equity, and inclusion (DEI) leadership in emergency medicine (EM) by identifying the prevalence of department DEI leadership positions, their demographics, and their job duty characteristics. Methods: We disseminated an electronic survey from April to July 2022 to Society for Academic Emergency Medicine (SAEM) Association of Academic Chairs of Emergency Medicine, Academy for Diversity and Inclusion in Emergency Medicine, and the Equity and Inclusion Committee to identify department DEI leads. From July to August 2022, a 45-question survey was sent to all identified DEI leaders on individual characteristics, DEI experience, and DEI lead job description. Results: We received a response from 79 out of 120 academic EM departments identified (65.8%). Of the responding institutions, 59 (74.7%) reported a DEI leader. A total of 74.6% of these DEI leaders responded at least partially to our survey and 57.6% responded in full. The most common titles were vice/associate chair of DEI (34.4%), director of DEI (28.1%), and DEI committee chair (18.8%). Most respondents (84.4%) were the inaugural DEI lead in their department and 84.4% of respondents did not have a formal DEI role in their department previously. On average, respondents have had their DEI title for 2 years (range 0-7 years) with an average of 7 years (range 0-30 years) of experience performing DEI work. Many (63.4%) do not receive any funded effort for their DEI roles. Most DEI leads were not tenure track (72.2%) and most commonly at the rank of assistant professor (47.2%) followed by associate professor (33.3%), full professor (16.7%), and instructor (2.8%). Conclusions: This is the first known study to assess the characteristics of DEI department leaders in EM. EM DEI leadership positions are new, common, and led by diverse personal identities and are often not funded. Future directions could gain qualitative insight into this workforce to guide best practices in EM DEI leadership.

Understanding clerkship experiences in emergency medicine and their potential influence on specialty selection: A qualitative study.

Objectives: The specialty of emergency medicine (EM) is experiencing a significant decrease in student interest. In addition, women are historically underrepresented within the specialty at all levels of training and practice. We sought to understand how clinical experiences and perceptions of EM influence specialty selection by medical students, particularly women. Methods: Using a constructivist grounded theory approach, we analyzed semistructured interviews with senior medical students who considered EM as a specialty. We used purposive sampling to recruit from diverse learning environments and represent a variety of experiences. Participants reflected on their specialty selection process and experiences in EM including their perceived acceptance in the work environment. Results: Twenty-five medical students from 11 geographically diverse schools participated. A total of 68% (17/25) identified as women. The majority (21/25, 84%) planned on applying to EM residency. We identified four major themes: (1) distressing interpersonal interactions with patients and the ED care team negatively affect students; (2) EM culture includes behaviors that are perceived as exclusionary; (3) beliefs about the attributes of an ideal EM physician and the specialty itself have a gendered nature; and (4) ease of access to mentors, representation, and early exposure to EM environment increased interest in specialty. Conclusions: Our participants express that EM causes challenges for students to accept the norms of behavior in the field, which is an essential element in joining a group and professional identity formation. In addition, we raise concern that gendered perceptions and language may send exclusionary environmental cues that may negatively impact recruitment of a diverse physician workforce.

Improving Graduate Medical Education by Aggregating Data Across the Medical Education Continuum.

ABSTRACT: Meaningful improvements to graduate medical education (GME) have been achieved in recent decades, yet many GME improvement pilots have been small trials without rigorous outcome measures and with limited generalizability. Thus, lack of access to large-scale data is a key barrier to generating empiric evidence to improve GME. In this article, the authors examine the potential of a national GME data infrastructure to improve GME, review the output of 2 national workshops on this topic, and propose a path toward achieving this goal.The authors envision a future where medical education is shaped by evidence from rigorous research powered by comprehensive, multi-institutional data. To achieve this goal, premedical education, undergraduate medical education, GME, and practicing physician data must be collected using a common data dictionary and standards and longitudinally linked using unique individual identifiers. The envisioned data infrastructure could provide a foundation for evidence-based decisions across all aspects of GME and help optimize the education of individual residents.Two workshops hosted by the National Academies of Sciences, Engineering, and Medicine Board on Health Care Services explored the prospect of better using GME data to improve education and its outcomes. There was broad consensus about the potential value of a longitudinal data infrastructure to improve GME. Significant obstacles were also noted.Suggested next steps outlined by the authors include producing a more complete inventory of data already being collected and managed by key medical education leadership organizations, pursuing a grass-roots data sharing pilot among GME-sponsoring institutions, and formulating the technical and governance frameworks needed to aggregate data across organizations.The power and potential of big data is evident across many disciplines, and the authors believe that harnessing the power of big data in GME is the best next step toward advancing evidence-based physician education.

The Purpose, Design, and Promise of Medical Education Research Labs.

Medical education researchers are often subject to challenges that include lack of funding, collaborators, study subjects, and departmental support. The construct of a research lab provides a framework that can be employed to overcome these challenges and effectively support the work of medical education researchers; however, labs are relatively uncommon in the medical education field. Using case examples, the authors describe the organization and mission of medical education research labs contrasted with those of larger research team configurations, such as research centers, collaboratives, and networks. They discuss several key elements of education research labs: the importance of lab identity, the signaling effect of a lab designation, required infrastructure, and the training mission of a lab. The need for medical education researchers to be visionary and strategic when designing their labs is emphasized, start-up considerations and the likelihood of support for medical education labs is considered, and the degree to which department leaders should support such labs is questioned.

Association of a longitudinal, preclinical ultrasound curriculum with medical student performance.

INTRODUCTION: Point-of-care ultrasound (US) is used in clinical practice across many specialties. Ultrasound (US) curricula for medical students are increasingly common. Optimal timing, structure, and effect of ultrasound education during medical school remains poorly understood. This study aims to retrospectively determine the association between participation in a preclinical, longitudinal US curriculum and medical student academic performance. METHODS: All first-year medical students at a medical school in the Midwest region of the United States were offered a voluntary longitudinal US curriculum. Participants were selected by random lottery. The curriculum consisted of five three-hour hands on-sessions with matching asynchronous content covering anatomy and pathologic findings. Content was paired with organ system blocks in the standard first year curriculum at our medical school. Exam scores between the participating and non-participating students were compared to evaluate the objective impact of US education on performance in an existing curriculum. We hypothesized that there would be an association between participation in the curriculum and improved medical student performance. Secondary outcomes included shelf exam scores for the surgery, internal medicine, neurology clerkships and USMLE Step 1. A multivariable linear regression model was used to evaluate the association of US curriculum participation with student performance. Scores were adjusted for age, gender, MCAT percentile, and science or engineering degree. RESULTS: 76 of 178 students applied to participate in the curriculum, of which 51 were accepted. US curriculum students were compared to non-participating students (n = 127) from the same class. The US curriculum students performed better in cardiovascular anatomy (mean score 92.1 vs. 88.7, p = 0.048 after adjustment for multiple comparisons). There were no significant differences in cumulative cardiovascular exam scores, or in anatomy and cumulative exam scores for the gastroenterology and neurology blocks. The effect of US curriculum participation on cardiovascular anatomy scores was estimated to be an improvement of 3.48 points (95% CI 0.78-6.18). No significant differences were observed for USMLE Step 1 or clerkship shelf exams. There were no significant differences in either preclinical, clerkship or Step 1 score for the 25 students who applied and were not accepted and the 102 who did not apply. CONCLUSIONS: Participation in a preclinical longitudinal US curriculum was associated with improved exam performance in cardiovascular anatomy but not examination of other cardiovascular system concepts. Neither anatomy or comprehensive exam scores for neurology and gastrointestinal organ system blocks were improved.

Interest in Serving the Underserved: Role of Race, Gender, and Medical Specialty Plans.

Introduction: Medical students often express their plans to care for medically underserved populations, but little is known about how this interest remains during medical school (MS). This study examined how self-reported interest in working with medically underserved communities may change during MS training based on several student characteristics. Methods: A secondary data analysis of all student records in the Electronic Residency Application Service (ERAS) from 2005 to 2010 is presented. Predictors included gender, under-represented in medicine (URiM) status, age, academic metrics, career interest, and medical specialty choice. Outcomes included interest in caring for medically underserved populations when entering MS, graduating MS, and graduating MS controlling for entering interest. Results: The total population included 6890 student records (49.5% women and 18.2% URiM). Women had a higher likelihood of being interested in practicing in underserved communities when entering and graduating MS (odds ratio [OR] 1.55, 95% confidence interval [CI] 1.37-1.77; OR 1.24, 95% CI 1.09-1.40). For all outcomes, URiM students had a higher likelihood of planning on a career with underserved populations compared with their non-URiM peers. Compared with Emergency Medicine, Internal Medicine/Pediatrics and Family Medicine had a higher likelihood of plans to work with underserved populations upon entering, graduating, and at graduation controlling for entering interest. Discussion: Gender, race, and specialty choice all had meaningful associations with a student's plans on practice in an underserved community. This study's findings can help support efforts to improve MS diversity nationally and drive study on cultural effects embedded within medical specialty identity.

A Critical Disconnect: Residency Selection Factors Lack Correlation With Intern Performance.

BACKGROUND: Emergency medicine (EM) residency programs want to employ a selection process that will rank best possible applicants for admission into the specialty. OBJECTIVE: We tested if application data are associated with resident performance using EM milestone assessments. We hypothesized that a weak correlation would exist between some selection factors and milestone outcomes. METHODS: Utilizing data from 5 collaborating residency programs, a secondary analysis was performed on residents trained from 2013 to 2018. Factors in the model were gender, underrepresented in medicine status, United States Medical Licensing Examination Step 1 and 2 Clinical Knowledge (CK), Alpha Omega Alpha (AOA), grades (EM, medicine, surgery, pediatrics), advanced degree, Standardized Letter of Evaluation global assessment, rank list position, and controls for year assessed and program. The primary outcomes were milestone level achieved in the core competencies. Multivariate linear regression models were fitted for each of the 23 competencies with comparisons made between each model's results. RESULTS: For the most part, academic performance in medical school (Step 1, 2 CK, grades, AOA) was not associated with residency clinical performance on milestones. Isolated correlations were found between specific milestones (eg, higher surgical grade increased wound care score), but most had no correlation with residency performance. CONCLUSIONS: Our study did not find consistent, meaningful correlations between the most common selection factors and milestones at any point in training. This may indicate our current selection process cannot consistently identify the medical students who are most likely to be high performers as residents.

Predicting Medical School Enrollment Behavior: Comparing an Enrollment Management Model to Expert Human Judgment.

PURPOSE: Medical school admissions committees are tasked with fulfilling the values of their institutions through careful recruitment. Making accurate predictions regarding enrollment behavior of admitted students is critical to intentionally formulating class composition and impacts long-term physician representation. The predictive accuracy and potential advantages of employing an enrollment predictive model in medical school admissions compared with expert human judgment have not been tested. METHOD: The enrollment management-based predictive model previously generated using historical data was employed to provide a predicted enrollment percentage for each admitted student in the 2016-2017 application pool (N = 352). Concurrently, the human expert created a predicted enrollment percentage for each applicant while blinded to the values generated by the model. An absolute error for each applicant for both approaches was calculated. Statistical significance between approaches (expert vs. enrollment model) was assessed using t tests. RESULTS: The enrollment management approach was noninferior to expert prediction in all cases (P < .05) with a superior correct classification rate (77.7% vs. 71.2%). When considering subgroup analyses for specific populations of potential importance in recruiting (underrepresented in medicine, female, and in-state applicants), the enrollment management predictions were statistically more accurate (P < .05). CONCLUSIONS: Examining a single admitted class, the enrollment predictions using the enrollment management model were at least as accurate as the expert human estimates, and in specific populations of interest more accurate. This information can be readily exported for a real-time dashboard system to drive recruitment behaviors.

Choosing emergency medicine: Influences on medical students' choice of emergency medicine.

BACKGROUND: Relatively little is understood about which factors influence students' choice of specialty and when learners ultimately make this decision. OBJECTIVE: The objective is to understand how experiences of medical students relate to the timing of selection of Emergency Medicine (EM) as a specialty. Of specific interest were factors such as how earlier and more positive specialty exposure may impact the decision-making process of medical students. METHODS: A cross-sectional survey study of EM bound 4th year US medical students (MD and DO) was performed exploring when and why students choose EM as their specialty. An electronic survey was distributed in March 2015 to all medical students who applied to an EM residency at 4 programs representing different geographical regions. Descriptive analyses and multinomial logistic regressions were performed. RESULTS: 793/1372 (58%) responded. Over half had EM experience prior to medical school. When students selected EM varied: 13.9% prior to, 50.4% during, and 35.7% after their M3 year. Early exposure, presence of an EM residency program, previous employment in the ED, experience as a pre-hospital provider, and completion of an M3 EM clerkship were associated with earlier selection. Delayed exposure to EM was associated with later selection of EM. CONCLUSIONS: Early exposure and prior life experiences were associated with choosing EM earlier in medical school. The third year was identified as the most common time for definitively choosing the specialty.

Competency-based education: programme design and challenges to implementation.

CONTEXT: Competency-based education (CBE) has been widely cited as an educational framework for medical students and residents, and provides a framework for designing educational programmes that reflect four critical features: a focus on outcomes, an emphasis on abilities, a reduction of emphasis on time-based training, and promotion of learner centredness. Each of these features has implications and potential challenges for implementing CBE. METHODS: As an experiment in CBE programme design and implementation, the University of Michigan Master of Health Professions Education (UM-MHPE) degree programme was examined for lessons to be learned when putting CBE into practice. The UM-MHPE identifies 12 educational competencies and 20 educational entrustable professional activities (EPAs) that serve as the vehicle for both learning and assessment. The programme also defines distinct roles of faculty members as assessors, mentors and subject-matter experts focused on highly individualised learning plans adapted to each learner. CONCLUSIONS: Early experience with implementing the UM-MHPE indicates that EPAs and competencies can provide a viable alternative to traditional courses and a vehicle for rigorous assessment. A high level of individualisation is feasible but carries with it significant costs and makes intentional community building essential. Most significantly, abandoning a time-based framework is a difficult innovation to implement in a university structure that is predicated on time-based education.

Enrollment Management in Medical School Admissions: A Novel Evidence-Based Approach at One Institution.

PURPOSE: In higher education, enrollment management has been developed to accurately predict the likelihood of enrollment of admitted students. This allows evidence to dictate numbers of interviews scheduled, offers of admission, and financial aid package distribution. The applicability of enrollment management techniques for use in medical education was tested through creation of a predictive enrollment model at the University of Michigan Medical School (U-M). METHOD: U-M and American Medical College Application Service data (2006-2014) were combined to create a database including applicant demographics, academic application scores, institutional financial aid offer, and choice of school attended. Binomial logistic regression and multinomial logistic regression models were estimated in order to study factors related to enrollment at the local institution versus elsewhere and to groupings of competing peer institutions. A predictive analytic "dashboard" was created for practical use. RESULTS: Both models were significant at P < .001 and had similar predictive performance. In the binomial model female, underrepresented minority students, grade point average, Medical College Admission Test score, admissions committee desirability score, and most individual financial aid offers were significant (P < .05). The significant covariates were similar in the multinomial model (excluding female) and provided separate likelihoods of students enrolling at different institutional types. CONCLUSIONS: An enrollment-management-based approach would allow medical schools to better manage the number of students they admit and target recruitment efforts to improve their likelihood of success. It also performs a key institutional research function for understanding failed recruitment of highly desirable candidates.

Assessment challenges in competency-based education: A case study in health professions education.

There is a growing demand for health sciences faculty with formal training in education. Addressing this need, the University of Michigan Medical School created a Master in Health Professions Education (UM-MHPE). The UM-MHPE is a competency-based education (CBE) program targeting professionals. The program is individualized and adaptive to the learner's situation using personal mentoring. Critical to CBE is an assessment process that accurately and reliably determines a learner's competence in educational domains. The program's assessment method has two principal components: an independent assessment committee and a learner repository. Learners submit evidence of competence that is evaluated by three independent assessors. The assessments are presented to an Assessment Committee who determines whether the submission provides evidence of competence. The learner receives feedback on the submission and, if needed, the actions needed to reach competency. During the program's first year, six learners presented 10 submissions for review. Assessing learners in a competency-based program has created challenges; setting standards that are not readily quantifiable is difficult. However, we argue it is a more genuine form of assessment and that this process could be adapted for use within most competency-based formats. While our approach is demanding, we document practical learning outcomes that assess competence.

What is the Prevalence and Success of Remediation of Emergency Medicine Residents?

INTRODUCTION: The primary objective of this study was to determine the prevalence of remediation, competency domains for remediation, the length, and success rates of remediation in emergency medicine (EM). METHODS: We developed the survey in Surveymonkey™ with attention to content and response process validity. EM program directors responded how many residents had been placed on remediation in the last three years. Details regarding the remediation were collected including indication, length and success. We reported descriptive data and estimated a multinomial logistic regression model. RESULTS: We obtained 126/158 responses (79.7%). Ninety percent of programs had at least one resident on remediation in the last three years. The prevalence of remediation was 4.4%. Indications for remediation ranged from difficulties with one core competency to all six competencies (mean 1.9). The most common were medical knowledge (MK) (63.1% of residents), patient care (46.6%) and professionalism (31.5%). Mean length of remediation was eight months (range 1-36 months). Successful remediation was 59.9% of remediated residents; 31.3% reported ongoing remediation. In 8.7%, remediation was deemed "unsuccessful." Training year at time of identification for remediation (post-graduate year [PGY] 1), longer time spent in remediation, and concerns with practice-based learning (PBLI) and professionalism were found to have statistically significant association with unsuccessful remediation. CONCLUSION: Remediation in EM residencies is common, with the most common areas being MK and patient care. The majority of residents are successfully remediated. PGY level, length of time spent in remediation, and the remediation of the competencies of PBLI and professionalism were associated with unsuccessful remediation.

PROGNOSTIC VALUE OF THE MULTIPLE MINI-INTERVIEW FOR EMERGENCY MEDICINE RESIDENCY PERFORMANCE.

BACKGROUND: The Multiple Mini-Interview (MMI) uses short, structured contacts, and is known to predict medical school success better than traditional interviews and application materials. Its utility in Emergency Medicine residency selection is untested. OBJECTIVES: We investigate whether it provides additional information regarding future first-year resident performance that can be useful in resident selection. METHODS: From three Emergency Medicine residency programs, 71 interns in their first month completed an MMI developed to focus on desirable resident characteristics. Application data were reviewed. First-year resident performance assessments covering the American Council for Graduate Medical Education (ACGME) core competencies, along with professionalism and performance concerns, were obtained. Multiple logistic regressions were employed and MMI correlations were compared with program rank lists and typical selection factors. RESULTS: An individual's score on the MMI correlated with overall performance (p < 0.05) in single logistic regression. MMI correlated with ACGME individual competencies patient care and procedural skills at a less robust level (p < 0.1), but not with any other outcomes. Rank list position correlated with the diagnostic skill competency (p < 0.05), but no others. Traditional selection factors correlated with overall performance, disciplinary action, patient care, medical knowledge, and diagnostic skills (p < 0.05). MMI was not correlated significantly with the outcomes when included in multiple ordinal logistic regression with other selection factors. CONCLUSIONS: MMI scores correlate with overall performance, but are not statistically significant when other traditional selection factors were considered. The MMI process seems potentially superior to program rank list at correlating with first-year performance. The MMI may provide additional benefit when examined using a larger and more diverse sample.

The multiple mini-interview for emergency medicine resident selection.

BACKGROUND: The Multiple Mini-Interview (MMI) uses multiple, short-structured contacts to evaluate communication and professionalism. It predicts medical school success better than the traditional interview and application. Its acceptability and utility in emergency medicine (EM) residency selection are unknown. OBJECTIVE: We theorized that participants would judge the MMI equal to a traditional unstructured interview and it would provide new information for candidate assessment. METHODS: Seventy-one interns from 3 programs in the first month of training completed an eight-station MMI focused on EM topics. Pre- and post-surveys assessed reactions. MMI scores were compared with application data. RESULTS: EM grades correlated with MMI performance (F[1, 66] = 4.18; p < 0.05) with honors students having higher scores. Higher third-year clerkship grades were associated with higher MMI performance, although this was not statistically significant. MMI performance did not correlate with match desirability and did not predict most other components of an application. There was a correlation between lower MMI scores and lower global ranking on the Standardized Letter of Recommendation. Participants preferred a traditional interview (mean difference = 1.36; p < 0.01). A mixed format (traditional interview and MMI) was preferred over a MMI alone (mean difference = 1.1; p < 0.01). MMI performance did not significantly correlate with preference for the MMI. CONCLUSIONS: Although the MMI alone was viewed less favorably than a traditional interview, participants were receptive to a mixed-methods interview. The MMI does correlate with performance on the EM clerkship and therefore can measure important abilities for EM success. Future work will determine whether MMI performance predicts residency performance.