The Reliability of the Resuscitation Assessment Tool (RAT) in Assessing Emergency Medicine Resident Competence in Pediatric Resuscitation Scenarios: A Prospective Observational Pilot Study.

Introduction Emergency medicine (EM) postgraduate medical education in Canada has transitioned from traditional time-based training to competency-based medical education (CBME). In order to promote residents through stages of training, simulated assessments are needed to evaluate residents in high-stakes but low-frequency medical emergencies. There remains a gap in the literature pertaining to the use of evaluative tools in simulation, such as the Resuscitation Assessment Tool (RAT) in the new CBME curriculum design. Methods We completed a pilot study of resident physicians in one Canadian EM training program to evaluate the effectiveness and reliability of a simulation-based RAT for pediatric resuscitation. We recorded 10 EM trainees completing simulated scenarios and had nine EM physicians use the RAT tool to evaluate their performances. Generalizability theory was used to evaluate the reliability of the RAT tool. Results The mean RAT score for the management of pediatric myocarditis, cardiac arrest, and septic shock (appendicitis) across raters was 3.70, 3.73, and 4.50, respectively. The overall generalizability coefficient for testing simulated pediatric performance competency was 0.77 for internal consistency and 0.75 for absolute agreement. The performance of senior participants was superior to that of junior participants in the management of pediatric myocarditis (p = 0.01) but not statistically significant in the management of pediatric septic shock (p=0.77) or cardiac arrest (p =0.61). Conclusion Overall, our findings suggest that with an appropriately chosen simulated scenario, the RAT tool can be used effectively for the simulation of high-stakes and low-frequency scenarios for practice to enhance the new CBME curriculum in emergency medicine training programs.

Simulation in Canadian postgraduate emergency medicine training - a national survey.

OBJECTIVES: Simulation-based education (SBE) is an important training strategy in emergency medicine (EM) postgraduate programs. This study sought to characterize the use of simulation in FRCPC-EM residency programs across Canada. METHODS: A national survey was administered to residents and knowledgeable program representatives (PRs) at all Canadian FRCPC-EM programs. Survey question themes included simulation program characteristics, the frequency of resident participation, the location and administration of SBE, institutional barriers, interprofessional involvement, content, assessment strategies, and attitudes about SBE. RESULTS: Resident and PR response rates were 63% (203/321) and 100% (16/16), respectively. Residents reported a median of 20 (range 0-150) hours of annual simulation training, with 52% of residents indicating that the time dedicated to simulation training met their needs. PRs reported the frequency of SBE sessions ranging from weekly to every 6 months, with 15 (94%) programs having an established simulation curriculum. Two (13%) of the programs used simulation for resident assessment, although 15 (94%) of PRs indicated that they would be comfortable with simulation-based assessment. The most common PR-identified barriers to administering simulation were a lack of protected faculty time (75%) and a lack of faculty experience with simulation (56%). Interprofessional involvement in simulation was strongly valued by both residents and PRs. CONCLUSIONS: SBE is frequently used by Canadian FRCPC-EM residency programs. However, there exists considerable variability in the structure, frequency, and timing of simulation-based activities. As programs transition to competency-based medical education, national organizations and collaborations should consider the variability in how SBE is administered.

The writer's guide to education scholarship in emergency medicine: Education innovations (part 3).

OBJECTIVE: The scholarly dissemination of innovative medical education practices helps broaden the reach of this type of work, allowing scholarship to have an impact beyond a single institution. There is little guidance in the literature for those seeking to publish program evaluation studies and innovation papers. This study aims to derive a set of evidence-based features of high-quality reports on innovations in emergency medicine (EM) education. METHODS: We conducted a scoping review and thematic analysis to determine quality markers for medical education innovation reports, with a focus on EM. A search of MEDLINE, EMBASE, ERIC, and Google Scholar was augmented by a hand search of relevant publication guidelines, guidelines for authors, and website submission portals from medical education and EM journals. Study investigators reviewed the selected articles, and a thematic analysis was conducted. RESULTS: Our search strategy identified 14 relevant articles from which 34 quality markers were extracted. These markers were grouped into seven important themes: goals and need for innovation, preparation, innovation development, innovation implementation, evaluation of innovation, evidence of reflective practice, and reporting and dissemination. In addition, multiple outlets for the publication of EM education innovations were identified and compiled. CONCLUSION: The publication and dissemination of innovations are critical for the EM education community and the training of health professionals. We anticipate that our list of innovation report quality markers will be used by EM education innovators to support the dissemination of novel educational practices.

CAEP 2016 Academic Symposium: A Writer's Guide to Key Steps in Producing Quality Medical Education Scholarship.

A key skill for successful clinician educators is the effective dissemination of scholarly innovations and research. Although there are many ways to disseminate scholarship, the most accepted and rewarded form of educational scholarship is publication in peer-reviewed journals. This paper provides direction for emergency medicine (EM) educators interested in publishing their scholarship via traditional peer-reviewed avenues. It builds upon four literature reviews that aggregated recommendations for writing and publishing high-quality quantitative and qualitative research, innovations, and reviews. Based on the findings from these literature reviews, the recommendations were prioritized for importance and relevance to novice clinician educators by a broad community of medical educators. The top items from the expert vetting process were presented to the 2016 Canadian Association of Emergency Physicians (CAEP) Academic Symposium Consensus Conference on Education Scholarship. This community of EM educators identified the highest yield recommendations for junior medical education scholars. This manuscript elaborates upon the top recommendations identified through this consensus-building process.

Comparison of Simulation-based Resuscitation Performance Assessments With In-training Evaluation Reports in Emergency Medicine Residents: A Canadian Multicenter Study.

OBJECTIVE: Simulation stands to serve an important role in modern competency-based programs of assessment in postgraduate medical education. Our objective was to compare the performance of individual emergency medicine (EM) residents in a simulation-based resuscitation objective structured clinical examination (OSCE) using the Queen's Simulation Assessment Tool (QSAT), with portfolio assessment of clinical encounters using a modified in-training evaluation report (ITER) to understand in greater detail the inferences that may be drawn from a simulation-based OSCE assessment. METHODS: A prospective observational study was employed to explore the use of a multicenter simulation-based OSCE for evaluation of resuscitation competence. EM residents from five Canadian academic sites participated in the OSCE. Video-recorded performances were scored by blinded raters using the scenario-specific QSATs with domain-specific anchored scores (primary assessment, diagnostic actions, therapeutic actions, communication) and a global assessment score (GAS). Residents' portfolios were evaluated using a modified ITER subdivided by CanMEDS roles (medical expert, communicator, collaborator, leader, health advocate, scholar, and professional) and a GAS. Correlational and regression analyses were performed comparing components of each of the assessment methods. RESULTS: Portfolio review and ITER scoring was performed for 79 residents participating in the simulation-based OSCE. There was a significant positive correlation between total OSCE and ITER scores (r = 0.341). The strongest correlations were found between ITER medical expert score and each of the OSCE GAS (r = 0.420), communication (r = 0.443), and therapeutic action (r = 0.484) domains. ITER medical expert was a significant predictor of OSCE total (p = 0.002). OSCE therapeutic action was a significant predictor of ITER total (p = 0.02). CONCLUSIONS: Simulation-based resuscitation OSCEs and portfolio assessment captured by ITERs appear to measure differing aspects of competence, with weak to moderate correlation between those measures of conceptually similar constructs. In a program of competency-based assessment of EM residents, a simulation-based OSCE using the QSAT shows promise as a tool for assessing medical expert and communicator roles.