A Novel Tool for Assessment of Emergency Medicine Resident Skill in Determining Diagnosis and Management for Emergent Electrocardiograms: A Multicenter Study.

BACKGROUND: Reading emergent electrocardiograms (ECGs) is one of the emergency physician's most crucial tasks, yet no well-validated tool exists to measure resident competence in this skill. OBJECTIVES: To assess validity of a novel tool measuring emergency medicine resident competency for interpreting, and responding to, critical ECGs. In addition, we aim to observe trends in this skill for resident physicians at different levels of training. METHODS: This is a multi-center, prospective study of postgraduate year (PGY) 1-4 residents at five emergency medicine (EM) residency programs in the United States. An assessment tool was created that asks the physician to identify either the ECG diagnosis or the best immediate management. RESULTS: One hundred thirteen EM residents from five EM residency programs submitted completed assessment surveys, including 43 PGY-1s, 33 PGY-2s, and 37 PGY-3/4s. PGY-3/4s averaged 74.6% correct (95% confidence interval [CI] 70.9-78.4) and performed significantly better than PGY-1s, who averaged 63.2% correct (95% CI 58.0-68.3). PGY-2s averaged 69.0% (95% CI 62.2-73.7). Year-to-year differences were more pronounced in management than in diagnosis. CONCLUSIONS: Residency training in EM seems to be associated with improved ability to interpret "critical" ECGs as measured by our assessment tool. This lends validity evidence for the tool by correlating with a previously observed association between residency training and improved ECG interpretation. Resident skill in ECG interpretation remains less than ideal. Creation of this sort of tool may allow programs to assess resident performance as well as evaluate interventions designed to improve competency.

Faculty Prediction of In-Training Examination Scores of Emergency Medicine Residents: A Multicenter Study.

BACKGROUND: The Emergency Medicine In-Training Examination (EMITE) is one of the few validated instruments for medical knowledge assessment of emergency medicine (EM) residents. The EMITE is administered only once annually, with results available just 2 months before the end of the academic year. An earlier predictor of EMITE scores would be helpful for educators to institute timely remediation plans. A previous single-site study found that only 69% of faculty predictions of EMITE scores were accurate. OBJECTIVE: The goal of this article was to measure the accuracy with which EM faculty at five residency programs could predict EMITE scores for resident physicians. METHODS: We asked EM faculty at five different residency programs to predict the 2014 EMITE scores for all their respective resident physicians. The primary outcome was prediction accuracy, defined as the proportion of predictions within 6% of the actual scores. The secondary outcome was prediction precision, defined as the mean deviation of predictions from the actual scores. We assessed faculty background variables for correlation with the two outcomes. RESULTS: One hundred and eleven faculty participated in the study (response rate 68.9%). Mean prediction accuracy for all faculty was 60.0%. Mean prediction precision was 6.3%. Participants were slightly more accurate at predicting scores of noninterns compared to interns. No faculty background variable correlated with the primary or secondary outcomes. Eight participants predicted scores with high accuracy (>80%). CONCLUSIONS: In this multicenter study, EM faculty possessed only moderate accuracy at predicting resident EMITE scores. A very small subset of faculty members is highly accurate.

Emergency Medicine Faculty Are Poor at Predicting Burnout in Individual Trainees: An Exploratory Study.

OBJECTIVE: Burnout is common among emergency medicine (EM) physicians, and it is prevalent even among EM trainees. Recently proposed Accreditation Council for Graduate Medical Education requirements encourage faculty to alert residency leadership when trainees display signs of burnout. It remains uncertain how trainees experiencing burnout can be reliably identified. We examined if EM faculty advisers at one institution can accurately predict burnout in their EM resident advisees. METHODS: In this cross-sectional, exploratory study at a single institution, we measured EM trainee burnout using the Maslach Burnout Inventory through a confidential, electronic survey. We subsequently asked EM faculty to predict if their designated advisees were experiencing burnout through a separate confidential, electronic survey. Burnout results were dichotomized from each survey and compared using a 2 × 2 contingency table and Fisher's exact test. RESULTS: Thirty-six of 54 (66.7%) eligible EM trainees completed the burnout assessment. Eleven of 19 (57.9%) eligible faculty advisers completed trainee burnout predictions, resulting in 30 of 54 (55.6%) trainees who completed the burnout assessment and had a faculty burnout prediction. Trainees reported an overall burnout rate of 70.0% (95% confidence interval [CI] = 53.6% to 86.4%). Cumulative faculty predictions of trainee burnout resulted in an overall burnout rate of 16.7% (95% CI = -5.3% to 38.7%). The sensitivity and specificity of faculty predictions of trainee burnout were 19.1% (95% CI = 5.5% to 41.9%) and 88.9% (95% CI = 51.8% to 99.7%), respectively. Faculty prediction of trainee burnout had a positive predictive value of 80.0% (95% CI = 28.4% to 99.5%) and a negative predictive value of 32.0% (95% CI = 15.0% to 53.5). The difference between trainees' reported rate of burnout and faculty predictions of trainee burnout was significant (p < 0.001). CONCLUSION: Emergency medicine faculty prediction of trainee burnout was poor. Education on recognizing burnout and other methods of identifying trainee burnout may be necessary.

Supplemental Milestones for Emergency Medicine Residency Programs: A Validation Study.

INTRODUCTION: Emergency medicine (EM) residency programs may be 36 or 48 months in length. The Residency Review Committee for EM requires that 48-month programs provide educational justification for the additional 12 months. We developed additional milestones that EM training programs might use to assess outcomes in domains that meet this accreditation requirement. This study aims to assess for content validity of these supplemental milestones using a similar methodology to that of the original EM Milestones validation study. METHODS: A panel of EM program directors (PD) and content experts at two institutions identified domains of additional training not covered by the existing EM Milestones. This led to the development of six novel subcompetencies: "Operations and Administration," "Critical Care," "Leadership and Management," "Research," "Teaching and Learning," and "Career Development." Subject-matter experts at other 48-month EM residency programs refined the milestones for these subcompetencies. PDs of all 48-month EM programs were then asked to order the proposed milestones using the Dreyfus model of skill acquisition for each subcompetency. Data analysis mirrored that used in the original EM Milestones validation study, leading to the final version of our supplemental milestones. RESULTS: Twenty of 33 subjects (58.8%) completed the study. No subcompetency or individual milestone met deletion criteria. Of the 97 proposed milestones, 67 (69.1%) required no further editing and remained at the same level as proposed by the study authors. Thirty milestones underwent level changes: 15 (15.5%) were moved one level up and 13 (13.4%) were moved one level down. One milestone (1.0%) in "Leadership and Management" was moved two levels up, and one milestone in "Operations and Administration" was moved two levels down. One milestone in "Research" was ranked by the survey respondents at one level higher than that proposed by the authors; however, this milestone was kept at its original level assignment. CONCLUSION: Six additional subcompetencies were generated and assessed for content validity using the same methodology as was used to validate the current EM Milestones. These optional milestones may serve as an additional set of assessment tools that will allow EM residency programs to report these additional educational outcomes using a familiar milestone rubric.

Randomised controlled trial to assess the effect of a Just-in-Time training on procedural performance: a proof-of-concept study to address procedural skill decay.

BACKGROUND: A subset of high-risk procedures present significant safety threats due to their (1) infrequent occurrence, (2) execution under time constraints and (3) immediate necessity for patient survival. A Just-in-Time (JIT) intervention could provide real-time bedside guidance to improve high-risk procedural performance and address procedural deficits associated with skill decay. OBJECTIVE: To evaluate the impact of a novel JIT intervention on transvenous pacemaker (TVP) placement during a simulated patient event. METHODS: This was a prospective, randomised controlled study to determine the effect of a JIT intervention on performance of TVP placement. Subjects included board-certified emergency medicine physicians from two hospitals. The JIT intervention consisted of a portable, bedside computer-based procedural adjunct. The primary outcome was performance during a simulated patient encounter requiring TVP placement, as assessed by trained raters using a technical skills checklist. Secondary outcomes included global performance ratings, time to TVP placement, number of critical omissions and System Usability Scale scores (intervention only). RESULTS: Groups were similar at baseline across all outcomes. Compared with the control group, the intervention group demonstrated statistically significant improvement in the technical checklist score (11.45 vs 23.44, p<0.001, Cohen's d effect size 4.64), the global rating scale (2.27 vs 4.54, p<0.001, Cohen's d effect size 3.76), and a statistically significant reduction in critical omissions (2.23 vs 0.68, p<0.001, Cohen's d effect size -1.86). The difference in time to procedural completion was not statistically significant between conditions (11.15 min vs 12.80 min, p=0.12, Cohen's d effect size 0.65). System Usability Scale scores demonstrated excellent usability. CONCLUSION: A JIT intervention improved procedure perfromance, suggesting a role for JIT interventions in rarely performed procedures.

Assessing Team Leadership in Emergency Medicine: The Milestones and Beyond.

BACKGROUND: Team leadership is a critical skill for emergency medicine physicians that directly affects team performance and the quality of patient care. There exists a robust body of team science research supporting team leadership conceptual models and behavioral skill sets. However, to date, this work has not been widely incorporated into health care team leadership education. OBJECTIVE: This narrative review has 3 aims: (1) to synthesize the team science literature and to translate important concepts and models to health care team leadership; (2) to describe how team leadership is currently represented in the health care literature and in the Accreditation Council for Graduate Medical Education Milestones for emergency medicine; and (3) to propose a novel, evidence-based framework for the assessment of team leadership in emergency medicine. METHODS: We conducted a narrative review of the team science and health care literature. We summarized our findings and identified a list of team leadership behaviors that were then used to create a framework for team leadership assessment. RESULTS: Current health care team leadership measurement tools do not incorporate evidence-based models of leadership concepts from other established domains. The emergency medicine milestones include several team leadership behaviors as part of a larger resident evaluation program. However, they do not offer a comprehensive or cohesive representation of the team leadership construct. CONCLUSIONS: Despite the importance of team leadership to patient care, there is no standardized approach to team leadership assessment in emergency medicine. Based on the results of our review, we propose a novel team leadership assessment framework that is supported by the team science literature.

A novel online didactic curriculum helps improve knowledge acquisition among non-emergency medicine rotating residents.

OBJECTIVES: Rotating residents represent a significant proportion of housestaff in academic emergency departments (EDs), yet they rarely receive targeted didactic education during their emergency medicine (EM) rotations. The goals of this study were: 1) to determine the effectiveness of an online didactic curriculum in improving EM knowledge among rotating residents and 2) to assess rotating resident satisfaction with this curriculum. METHODS: The authors created an online lecture series of six EM subject areas targeted to rotating residents called the Northwestern University Rotating Resident Curriculum (NURRC). All rotating residents at the study site were eligible, written consent was obtained, and the study was approved by the institutional review board. Consenting participants were pretested with a 42-question multiple-choice examination and then randomized to two groups: one with access to the NURRC during the first 2 weeks of the rotation (experimental) and one without (control). Halfway through the rotation, all participants were post-tested with a different multiple-choice examination, and the controls were then granted NURRC access. The primary outcome was the difference between pretest and posttest scores (score delta). The t-test was used to compare mean scores, and a linear regression model was used to determine the association of NURRC access on score delta after adjustment for pretest type and resident type. A postintervention survey was administered at the end of the rotation to assess satisfaction with the NURRC and collect suggestions for improvement. RESULTS: Fifty-four rotating residents were enrolled: 29 in the experimental group and 25 in the control group. There was no significant difference in pretest scores between the two groups. Mean score delta was 17.3% in the experimental group and 1.6% in the control group, an absolute difference of 15.7% (95% confidence interval [CI]=10% to 22%). After adjustment for resident type and pretest type, the only variable positively associated with the primary outcome was NURRC access. Third-year and preliminary-year internal medicine (IM) residents demonstrated the greatest absolute improvement in score delta when granted NURRC access. Eighty percent of the participants responded to the satisfaction survey. Over 80% of the survey respondents approved of each component lecture and of the NURRC overall. CONCLUSIONS: After exposure to an online didactic curriculum, rotating residents demonstrated a significant increase in EM knowledge and reported a high level of satisfaction with the didactic program.

Rotating resident didactics in the emergency department: a cross-sectional survey on current curricular practices.

OBJECTIVES: Rotating (non-emergency medicine [EM]) residents perform clinical rotations in many academic emergency departments (EDs). The primary objective of this work was to quantify characteristics of rotating residents and the didactic curricula offered to them during their EM rotations. Secondary objectives were to identify barriers to instituting such didactics and to establish ideal curricular contents. METHODS: A Web-based survey was administered by e-mail to residency directors of all U.S. allopathic EM residency programs. Consent was obtained in the first part of the survey, and the study was deemed exempt from full review by the institutional review board. Questions solicited information regarding type and quantity of rotating residents in their main EDs, the "didactic educational format" available to rotating residents, and ideal and actual didactic curricular contents. Statistics were reported as proportions and means with 95% confidence intervals (CIs) and medians with interquartile ranges (IQRs). RESULTS: Surveys were sent to 143 programs, and the response rate was 71%. Ninety-nine percent of respondents had rotating residents in their EDs, and the median number per month was 4 (IQR = 3-6). Five percent of respondents had established didactic curricula specifically for rotating residents, and 64% sent them to either EM resident or medical student lectures. Thirty-one percent of programs reported no didactics, and 65% of these felt there was no need for such education. Resuscitation, trauma, and toxicology were cited as the most important subjects for actual and ideal curricula. CONCLUSIONS: Most academic EDs have rotating residents, but very few provide didactic education specific to their learning needs and almost a third provide no didactics.