Was it all worth it? A graduating resident perspective on CBME.

BACKGROUND: Our institution simultaneously transitioned all postgraduate specialty training programs to competency-based medical education (CBME) curricula. We explored experiences of CBME-trained residents graduating from five-year programs to inform the continued evolution of CBME in Canada. METHODS: We utilized qualitative description to explore residents' experiences and inform continued CBME improvement. Data were collected from fifteen residents from various specialties through focus groups, interviews, and written responses. The data were analyzed inductively, using conventional content analysis. RESULTS: We identified five overarching themes. Three themes provided insight into residents' experiences with CBME, describing discrepancies between the intentions of CBME and how it was enacted, challenges with implementation, and variation in residents' experiences. Two themes - adaptations and recommendations - could inform meaningful refinements for CBME going forward. CONCLUSIONS: Residents graduating from CBME training programs offered a balanced perspective, including criticism and recognition of the potential value of CBME when implemented as intended. Their experiences provide a better understanding of residents' needs within CBME curricula, including greater balance and flexibility within programs of assessment and curricula. Many challenges that residents faced with CBME could be alleviated by greater accountability at program, institutional, and national levels. We conclude with actionable recommendations for addressing residents' needs in CBME.

"I am the doctor": gender-based bias within the clinical practice of emergency medicine in Canada-a thematic analysis of physician and trainee interview data.

OBJECTIVES: While women comprise about half of current Canadian medical students and physicians, only 31% of emergency medicine physicians identify as women and women trainees are less likely to express interest in emergency medicine compared to men. Gender-based bias continues to negatively impact the career choice, progress, and well-being of women physicians/trainees. Although instances of gender-based bias are well documented within other medical specialties, there remains a gap in the literature addressing the role of gender specific to the Canadian emergency medicine clinical environment. METHODS: Using a qualitative study with a thematic analytical approach, participants were purposively and snowball sampled from a cross-section of centers across Canada and included emergency medicine attending physicians and trainees. A thematic analysis using an inductive and deductive approach was undertaken. All data were double coded to improve study trustworthiness. Descriptive statistics were used to characterize the study population. RESULTS: Thirty-four individuals (17 woman-identifying and 17 man-identifying) from 10 different institutions across 4 provinces in Canada participated in the study. Six themes were identified: (1) women experience gender bias in the form of microaggressions; (2) women experience imposter syndrome and question their role in the clinical setting; (3) more women provide patient care to women patients and vulnerable populations; (4) gender-related challenges with family planning and home responsibilities affect work-life balance; (5) allyship and sponsorship are important for the support and development of women physicians and trainees; and (6) women value discussing shared experiences with other women to debrief situations, find mentorship, and share advice. CONCLUSIONS: Gender inequity in emergency medicine affects women-identifying providers at all levels of training across Canada. Described experiences support several avenues to implement change against perceived gender bias that is focused on education, policy, and supportive spaces. We encourage institutions to consider these recommendations to achieve gender-equitable conditions in emergency medicine across Canada.

ABSTRAIT: OBJECTIFS: Bien que les femmes représentent environ la moitié des étudiants et des médecins en médecine au Canada, seulement 31 % des médecins d'urgence qui s'identifient comme des femmes et des femmes stagiaires sont moins susceptibles d'exprimer leur intérêt pour la médecine d'urgence que les hommes. Les préjugés fondés sur le sexe continuent d'avoir une incidence négative sur le choix de carrière, les progrès et le bien-être des femmes médecins/stagiaires. Bien que les cas de biais fondés sur le sexe soient bien documentés dans d'autres spécialités médicales, il reste une lacune dans la documentation traitant du rôle du sexe propre au milieu clinique de la médecine d'urgence au Canada. MéTHODES: À l'aide d'une étude qualitative avec une approche analytique thématique, les participants ont été échantillonnés à dessein et en boule de neige dans un échantillon représentatif de centres à travers le Canada et comprenaient des médecins urgentistes et des stagiaires. Une analyse thématique utilisant une approche inductive et déductive a été entreprise. Toutes les données ont été codées en double pour améliorer la fiabilité de l'étude. Des statistiques descriptives ont été utilisées pour caractériser la population étudiée. RéSULTATS: Trente-quatre personnes (17 femmes et 17 hommes) de 10 établissements différents de quatre provinces canadiennes ont participé à l'étude. Six thèmes ont été cernés : (1) les femmes sont victimes de préjugés sexistes sous la forme de microagressions; (2) les femmes sont victimes du syndrome d'imposteur et remettent en question leur rôle dans le milieu clinique; (3) plus de femmes prodiguent des soins aux patientes et aux populations vulnérables; (4) les défis liés au genre que posent la planification familiale et les responsabilités familiales ont une incidence sur l'équilibre entre le travail et la vie personnelle; (5) l'alliance et le parrainage sont importants pour le soutien et le perfectionnement des femmes médecins et stagiaires; (6) les femmes apprécient de discuter des expériences partagées avec d'autres femmes pour faire le point sur des situations, trouver du mentorat et partager des conseils. CONCLUSIONS: L'inégalité entre les sexes en médecine d'urgence touche les fournisseurs de soins qui identifient les femmes à tous les niveaux de formation au Canada. Les expériences décrites appuient plusieurs avenues pour mettre en œuvre des changements contre les préjugés sexistes perçus qui sont axés sur l'éducation, les politiques et les espaces de soutien. Nous encourageons les établissements à tenir compte de ces recommandations afin de parvenir à des conditions équitables entre les sexes en médecine d'urgence partout au Canada.

The Assessment Burden in Competency-Based Medical Education: How Programs Are Adapting.

Residents and faculty have described a burden of assessment related to the implementation of competency-based medical education (CBME), which may undermine its benefits. Although this concerning signal has been identified, little has been done to identify adaptations to address this problem. Grounded in an analysis of an early Canadian pan-institutional CBME adopter's experience, this article describes postgraduate programs' adaptations related to the challenges of assessment in CBME. From June 2019-September 2022, 8 residency programs underwent a standardized Rapid Evaluation guided by the Core Components Framework (CCF). Sixty interviews and 18 focus groups were held with invested partners. Transcripts were analyzed abductively using CCF, and ideal implementation was compared with enacted implementation. These findings were then shared back with program leaders, adaptations were subsequently developed, and technical reports were generated for each program. Researchers reviewed the technical reports to identify themes related to the burden of assessment with a subsequent focus on identifying adaptations across programs. Three themes were identified: (1) disparate mental models of assessment processes in CBME, (2) challenges in workplace-based assessment processes, and (3) challenges in performance review and decision making. Theme 1 included entrustment interpretation and lack of shared mindset for performance standards. Adaptations included revising entrustment scales, faculty development, and formalizing resident membership. Theme 2 involved direct observation, timeliness of assessment completion, and feedback quality. Adaptations included alternative assessment strategies beyond entrustable professional activity forms and proactive assessment planning. Theme 3 related to resident data monitoring and competence committee decision making. Adaptations included adding resident representatives to the competence committee and assessment platform enhancements. These adaptations represent responses to the concerning signal of significant burden of assessment within CBME being experienced broadly. The authors hope other programs may learn from their institution's experience and navigate the CBME-related assessment burden their invested partners may be facing.

Addressing microaggressions with simulation: a novel educational intervention.

There is an urgent need for education around equity, diversity, inclusivity, indigeneity, and accessibility (EDIIA). One important facet of this is gender-related microaggressions, which are a common occurrence in the emergency department. Most emergency medicine residents receive few opportunities to discuss, understand, and approach these occurrences in the clinical setting. To address this, we created a novel immersive session exploring gender-based microaggressions through a simulation experience followed by reflective teaching to foster allyship and practice tools for responding to microaggressions. An anonymous survey was subsequently distributed to elicit feedback, which was positive. After this successful pilot, next steps include creating sessions to address other microaggressions. Limitations include implicit biases of facilitators and ensuring that facilitators can engage in brave spaces and open conversations. Others trying to integrate gendered microaggression training into their EDIIA curricula could model our innovation.

RéSUMé: Il existe un besoin urgent d'éducation sur l'équité, la diversité, l'inclusivité, l'indigénéité et l'accessibilité (EDIIA). Les micro-agressions liées au genre, qui sont monnaie courante dans les services d'urgence, en sont une facette importante. La plupart des résidents en médecine d'urgence ont peu d'occasions de discuter, de comprendre et d'aborder ces événements en milieu clinique. Pour y remédier, nous avons créé une nouvelle session immersive explorant les micro-agressions basées sur le genre par le biais d'une expérience de simulation suivie d'un enseignement réflexif pour favoriser l'alliance et la pratique d'outils pour répondre aux micro-agressions. Un sondage anonyme a ensuite été distribué pour obtenir des commentaires, qui ont été positifs. Après ce projet pilote réussi, les prochaines étapes comprennent la création de séances pour aborder d'autres micro-agressions. Les limites comprennent les préjugés implicites des animateurs et la garantie que les animateurs peuvent s'engager dans des espaces courageux et des conversations ouvertes. D'autres personnes qui tentent d'intégrer une formation sur les micro-agressions sexistes dans leur programme EDIIA pourraient s'inspirer de notre innovation.

COVID-19 has heightened tensions between and exposed threats to core values of emergency medicine.

BACKGROUND: Professional culture is a powerful influence in emergency departments, but incompletely understood. Disasters magnify cultural realities, and as such the COVID-19 pandemic offered a unique opportunity to better understand emergency medicine (EM) values, practices, and beliefs. METHODS: We conducted a collaborative ethnography at a tertiary care center during the acute phase of the response to the threat of COVID-19 (March-May 2020). Collaborative ethnography is a method that partners directly with communities during design, data gathering, and analysis to study culture. An ED-based research team gathered data including field notes from 300 h of participant observation and informal interviews, 42 semi-structured interviews, and 57 departmental documents. Data were deductively coded using a previously generated framework for understanding EM culture. RESULTS: Each of seven core values from the original framework were identified in the dataset and further contextualized understanding of EM culture. COVID-19 exacerbated pre-existing tensions and threats to the core values of EM. For example, the desire to provide patient-centered care was impeded by strict visitor restrictions; the ability to treat life-threatening illness was impaired by new resuscitation room layouts and infection control procedures; and subtle changes in protocols had downstream impact on flow and the ability to balance needs and resources at a system level. The cultural values related to teams were protective and strengthened during this time. The pandemic exposed problems with the status quo, underscored inherent tensions between ED values, and highlighted threats to self-identity. CONCLUSION: COVID-19 has highlighted and compounded existing tensions and threats to the core values of EM, underscoring a critical mismatch between values and practice. Realignment of the realities of ED work with staff values is urgently needed.

RéSUMé: CONTEXTE: La culture professionnelle est une influence puissante dans les services d'urgence, mais elle est incomplètement comprise. Les catastrophes amplifient les réalités culturelles et, à ce titre, la pandémie de COVID-19 a offert une occasion unique de mieux comprendre les valeurs, les pratiques et les croyances de la médecine d'urgence (MU). MéTHODES: Nous avons mené une ethnographie collaborative dans un centre de soins tertiaires pendant la phase aiguë de la réponse à la menace du COVID-19 (mars-mai 2020). L'ethnographie collaborative est une méthode qui s'associe directement aux communautés pendant la conception, la collecte de données et l'analyse pour étudier la culture. Une équipe de recherche basée à l'urgence a recueilli des données, y compris des notes de terrain tirées de 300 heures d'observation des participants et d'entrevues informelles, de 42 entrevues semi-structurées et de 57 documents ministériels. Les données ont été codées de manière déductive à l'aide d'un cadre précédemment créé pour comprendre la culture de la MU. RéSULTATS: Chacune des sept valeurs fondamentales du cadre original a été identifiée dans l'ensemble de données et a permis de mieux comprendre la culture de la MU. COVID-19 a exacerbé les tensions préexistantes et les menaces qui pèsent sur les valeurs fondamentales de la MU. Par exemple, le désir de fournir des soins centrés sur le patient a été entravé par des restrictions strictes concernant les visiteurs ; la capacité de traiter des maladies potentiellement mortelles a été compromise par les nouvelles dispositions des salles de réanimation et des procédures de contrôle des infections ; et des changements subtils dans les protocoles ont eu un impact en aval sur le flux et la capacité à équilibrer les besoins et les ressources au niveau du système. Les valeurs culturelles liées aux équipes étaient protectrices et renforcées pendant cette période. La pandémie a mis en évidence les problèmes liés au statu quo, souligné les tensions inhérentes entre les valeurs des services d'urgence et mis en évidence les menaces pour l'identité personnelle. CONCLUSION: COVID-19 a mis en évidence et aggravé les tensions et les menaces existantes pour les valeurs fondamentales de la MU, soulignant un décalage critique entre les valeurs et la pratique. Il est urgent de réaligner les réalités du travail dans les services d'urgence sur les valeurs du personnel.

Creating a Competency-Based Medical Education Curriculum for Canadian Diagnostic Radiology Residency (Queen's Fundamental Innovations in Residency Education)-Part 2: Core of Discipline Stage.

PURPOSE: All postgraduate residency programs in Canada are transitioning to a competency-based medical education (CBME) model divided into 4 stages of training. Queen's University has been the first Canadian institution to mandate transitioning to CBME across all residency programs, including Diagnostic Radiology. This study describes the implementation of CBME with a focus on the third developmental stage, Core of Discipline, in the Diagnostic Radiology residency program at Queen's University. We describe strategies applied and challenges encountered during the adoption and implementation process in order to inform the development of other CBME residency programs in Diagnostic Radiology. METHODS: At Queen's University, the Core of Discipline stage was developed using the Royal College of Physicians and Surgeons of Canada's (RCPSC) competence continuum guidelines and the CanMEDS framework to create radiology-specific entrustable professional activities (EPAs) and milestones for assessment. New committees, administrative positions, and assessment strategies were created to develop these assessment guidelines. Currently, 2 cohorts of residents (n = 6) are enrolled in the Core of Discipline stage. RESULTS: EPAs, milestones, and methods of evaluation for the Core of Discipline stage are described. Opportunities during implementation included tracking progress toward educational objectives and increased mentorship. Challenges included difficulty meeting procedural volume requirements, inconsistent procedural tracking, improving feedback mechanisms, and administrative burden. CONCLUSION: The transition to a competency-based curriculum in an academic Diagnostic Radiology residency program is significantly resource and time intensive. This report describes challenges faced in developing the Core of Discipline stage and potential solutions to facilitate this process.

Creating a Competency-Based Medical Education Curriculum for Canadian Diagnostic Radiology Residency (Queen's Fundamental Innovations in Residency Education)-Part 1: Transition to Discipline and Foundation of Discipline Stages.

PURPOSE: The Royal College of Physicians and Surgeons of Canada (RCPSC) has mandated the transition of postgraduate medical training in Canada to a competency-based medical education (CBME) model divided into 4 stages of training. As part of the Queen's University Fundamental Innovations in Residency Education proposal, Queen's University in Canada is the first institution to transition all of its residency programs simultaneously to this model, including Diagnostic Radiology. The objective of this report is to describe the Queen's Diagnostic Radiology Residency Program's implementation of a CBME curriculum. METHODS: At Queen's University, the novel curriculum was developed using the RCPSC's competency continuum and the CanMEDS framework to create radiology-specific entrustable professional activities (EPAs) and milestones. In addition, new committees and assessment strategies were established. As of July 2015, 3 cohorts of residents (n = 9) have been enrolled in this new curriculum. RESULTS: EPAs, milestones, and methods of evaluation for the Transition to Discipline and Foundations of Discipline stages, as well as the opportunities and challenges associated with the implementation of a competency-based curriculum in a Diagnostic Radiology Residency Program, are described. Challenges include the increased frequency of resident assessments, establishing stage-specific learner expectations, and the creation of volumetric guidelines for case reporting and procedures. CONCLUSIONS: Development of a novel CBME curriculum requires significant resources and dedicated administrative time within an academic Radiology department. This article highlights challenges and provides guidance for this process.

Translating outcome frameworks to assessment programmes: Implications for validity.

OBJECTIVES: Competency-based medical education (CBME) requires that educators structure assessment of clinical competence using outcome frameworks. Although these frameworks may serve some outcomes well (e.g. represent eventual practice), translating these into workplace-based assessment plans may undermine validity and, therefore, trustworthiness of assessment decisions due to a number of competing factors that may not always be visible or their impact knowable. Explored here is the translation process from outcome framework to formative and summative assessment plans in postgraduate medical education (PGME) in three Canadian universities. METHODS: We conducted a qualitative study involving in-depth semi-structured interviews with leaders of PGME programmes involved in assessment and/or CBME implementation, with a focus on their assessment-based translational activities and evaluation strategies. Interviews were informed by Callon's theory of translation. Our analytical strategy involved directed content analysis, allowing us to be guided by Kane's validity framework, whilst still participating in open coding and analytical memo taking. We then engaged in axial coding to systematically explore themes across the dataset, various situations and our conceptual framework. RESULTS: Twenty-four interviews were conducted involving 15 specialties across three universities. Our results suggest: (i) using outcomes frameworks for assessment is necessary for good assessment but are also viewed as incomplete constructs; (ii) there are a number of social and practical negotiations with competing factors that displace validity as a core influencer in assessment planning, including implementation, accreditation and technology; and (iii) validity exists as threatened, uncertain and assumed due to a number of unchecked assumptions and reliance on surrogates. CONCLUSIONS: Translational processes in CBME involve negotiating with numerous influencing actors and institutions that, from an assessment perspective, provide challenges for assessment scientists, institutions and educators to contend with. These processes are challenging validity as a core element of assessment designs. Educators must reconcile these influences when preparing for or structuring validity arguments.

A systems approach for institutional CBME adoption at Queen's University.

The Royal College of Physicians and Surgeons of Canada (RCPSC) has begun the transition to Competency by Design (CBD), a new curricular model for residency education that 'ensure[s] competence, but teaches for excellence'. By 2022, all Canadian specialty programs are anticipated to have completed the CBD cohort process which includes workshops facilitated by a Royal College Clinician Educator. Queen's University in Ontario, Canada, was granted approval by the RCPSC to embark upon an accelerated path to competency-based medical education (CBME) for all our postgraduate specialties. This accelerated path allowed us to take an institutional approach for CBME implementation and ensure that all specialities were part of a system-wide change. Our unique institution-wide approach to CBD is the first of its kind across Canada. From both a theoretical and practical perspective we undertook CBME using a systems approach that allowed us to build the foundations for CBME, implement the change, and plan for sustainability. This has created opportunities to bridge and connect the various programs involved in the implementation of CBME on Queen's campus. The systems approach was an essential part of our strategy to develop a community dedicated to ensuring a successful CBME implementation.

Developing Academic Advisors and Competence Committees members: A community approach to developing CBME faculty leaders.

INTRODUCTION: Implementing competency-based medical education (CBME) at the institutional level poses many challenges including having to rapidly enable faculty to be facilitators and champions of a new curriculum which utilizes feedback, coaching, and models of programmatic assessment. This study presents the necessary competencies required for Academic Advisors (AA) and Competence Committee (CC) members, as identified in the literature and as perceived by faculty members at Queen's University. METHODS: This study integrated a review of available literature (n=26) yielding competencies that were reviewed by the authors followed by an external review consisting of CBME experts (n=5). These approved competencies were used in a cross-sectional community consultation survey distributed one year before (n=83) and one year after transitioning to CBME (n=144). FINDINGS: Our newly identified competencies are a useful template for other institutions. Academic Advisor competencies focused on mentoring and coaching, whereas Competence Committee member's competencies focused on integrating assessments and institutional policies. Competency discrepancies between stakeholder groups existing before the transition had disappeared in the post-implementation sample. CONCLUSIONS: We found value in taking an active community-based approach to developing and validating faculty leader competencies sooner rather than later when transitioning to CBME. The evolution of Competence Committees members and Academic Advisors requires the investment of specialized professional development and the sustained engagement of a collaborative community with shared concerns.

CONTEXTE: La mise en œuvre d'une formation médicale fondée sur les compétences (FMFC) au niveau institutionnel pose de nombreux défis, y compris de devoir permettre au corps professoral de devenir rapidement des facilitateurs et des champions d'un nouveau cursus qui fait appel à la rétroaction, à l'accompagnement et à l'évaluation programmatique. Cette étude présente les compétences nécessaires requises pour les conseillers pédagogiques(CP) et les membres des comités des compétences (CC), tel qu'identifié dans la littérature et comme perçues par le corps professoral à l'Université Queen. MÉTHODES: Cette étude a intégré une recension des écrits disponibles (n = 26) identifiantdes compétences, qui ont été évaluéespar les auteurs, suivie d'une évaluation externe composée d'experts de la FMFC (n = 5). Ces compétences approuvées ont été utilisées dans une consultation communautaire transversale distribuée une année avant(n = 83) et une année après la transition vers la FMFC (n = 144). RÉSULTATS: Nos compétences nouvellement déterminées représentent un modèle utile pour d'autres institutions. Les compétences d'un conseiller pédagogiquesont axées sur le mentorat et l'accompagnement, alors que les compétences des membres des comités des compétences sont axées sur l'intégration des évaluations et des politiques institutionnelles. Les divergences dans les compétences entre les parties prenantes existants avant la transition avaient disparu dans l'échantillon qui a suivi la mise en œuvre. CONCLUSIONS: Nous avons jugé utile d'adopter une approche active fondée sur lacommunauté pour élaborer et valider les compétences du corps professoral en position de leadership plus tôt que tard dans la transition vers la FMFC. L'évolution des membres des comités de compétences et des conseillers pédagogiquesnécessite un investissement dans un développement professoral spécialisé et un engagement soutenu d'une communauté collaborative qui présente des préoccupations communes.

Perceptions and barriers to competency-based education in Canadian postgraduate medical education.

RATIONALE: Competency-based education (CBE) is currently being implemented across Canadian postgraduate medical education programmes through Competence by Design (CBD).1 Queen's University received permission to initiate CBE in all programmes simultaneously starting in 2017; an institutional initiative termed Competency-based medical education (CBME).2 We describe our initial experiences to highlight perceptions and barriers and facilitate implementation at other centers. METHODS: Anonymous online surveys were administered to faculty and residents transitioning to CBE (138 respondents) including (a) Queen's programme leaders (Programme Directors and CBME Leads) [n = 27], (b) Queen's residents [n = 102], and (c) Canadian neurology programme directors [n = 9] and were analysed using descriptive and inferential statistical techniques. RESULTS: Perceptions were favourable (x = 3.55/5, SD = 0.71) and 81.6% perceived CBE enhanced training; however, perceptions were more favourable among faculty. Queen's programme leaders indicated that CBE did not improve their ability to provide negative feedback. Queen's residents did not perceive improved quality of feedback. National Canadian neurology programme directors did not perceive that their institutions had adequately prepared them. There was variability in barriers perceived across groups. Queen's programme leaders were concerned about resident initiative. Queen's residents felt that assessment selection and faculty responsiveness to feedback were barriers. Canadian neurology programme directors were concerned about access to information technology. RECOMMENDATIONS: Our results indicate that faculty were concerned about the reluctance of residents to actively participate in CBE, while residents were hesitant to assume such a role because of lack of familiarity and perceived benefit. This discrepancy indicates attention should be devoted to (a) institutional administrative/educational supports, (b) faculty development around feedback/assessment, and (c) resident development to foster ownership of their learning and familiarity with CBE.

A case for feedback and monitoring assessment in competency-based medical education.

PURPOSE: Within competency-based medical education, self-regulated learning (SRL) requires residents to leverage self-assessment and faculty feedback. We sought to investigate the potential for competency-based assessments to foster SRL by quantifying the relationship between faculty feedback and entrustment ratings as well as the congruence between faculty assessment and resident self-assessment. MATERIALS AND METHODS: We collected comments in (a) an emergency medicine objective structured clinical examination group (objective structured clinical examinations [OSCE] and emergency medicine OSCE group [EMOG]) and (b) a first-year resident multidisciplinary resuscitation "Nightmares" course assessment group (NCAG) and OSCE group (NOG). We assessed comments across five domains including Initial Assessment (IA), Diagnostic Action (DA), Therapeutic Action (TA), Communication (COM), and entrustment. Analyses included structured qualitative coding and (non)parametric and descriptive analyses. RESULTS: In the EMOG, faculty's positive comments in the entrustment domain corresponded to lower entrustment score Mean Ranks (MRs) for IA (<11.1), DA (<11.2), and entrustment (<11.6). In NOG, faculty's negative comments resulted in lower entrustment score MRs for TA (<11.8 and <10) and DA (<12.4), and positive comments resulted in higher entrustment score MRs for IA (>15.4) and COM (>17.6). In the NCAG, faculty's positive IA comments were negatively correlated with entrustment scores (ρ = -.27, P = .04). Across programs, faculty and residents made similar domain-specific comments 13% of the time. CONCLUSIONS: Minimal and inconsistent associations were found between narrative and numerical feedback. Performance monitoring accuracy and feedback should be included in assessment validation.

Starting to Think Like an Expert: An Analysis of Resident Cognitive Processes During Simulation-Based Resuscitation Examinations.

STUDY OBJECTIVE: Simulation is commonly used to teach crisis resource management skills and assess them in emergency medicine residents. However, our understanding of the cognitive processes underlying crisis resource management skills is limited because these processes are difficult to assess and describe. The objective of this study is to uncover and characterize the cognitive processes underlying crisis resource management skills and to describe how these processes vary between residents according to performance in a simulation-based examination. METHODS: Twenty-two of 24 eligible emergency medicine trainees from 1 tertiary academic center completed 1 or 2 resuscitation-based examinations in the simulation laboratory. Resident performance was assessed by a blinded expert using an entrustment-based scoring tool. Participants wore eye-tracking glasses that generated first-person video that was used to augment subsequent interviews led by an emergency medicine faculty member. Interviews were audio recorded and then transcribed. An emergent thematic analysis was completed with a codebook that was developed by 4 research assistants, with subsequent analyses conducted by the lead research assistant with input from emergency medicine faculty. Themes from high- and low-performing residents were subsequently qualitatively compared. RESULTS: Higher-performing residents were better able to anticipate, selectively attend to relevant information, and manage cognitive demands, and took a concurrent (as opposed to linear) approach to managing the simulated patient. CONCLUSION: The results provide new insights into residents' cognitive processes while managing simulated patients in an examination environment and how these processes vary with performance. More work is needed to determine how best to apply these findings to improve crisis resource management education.

A Core Components Framework for Evaluating Implementation of Competency-Based Medical Education Programs.

PURPOSE: The rapid adoption of competency-based medical education (CBME) provides an unprecedented opportunity to study implementation. Examining "fidelity of implementation"-that is, whether CBME is being implemented as intended-is hampered, however, by the lack of a common framework. This article details the development of such a framework. METHOD: A two-step method was used. First, a perspective indicating how CBME is intended to bring about change was described. Accordingly, core components were identified. Drawing from the literature, the core components were organized into a draft framework. Using a modified Delphi approach, the second step examined consensus amongst an international group of experts in CBME. RESULTS: Two different viewpoints describing how a CBME program can bring about change were found: production and reform. Because the reform model was most consistent with the characterization of CBME as a transformative innovation, this perspective was used to create a draft framework. Following the Delphi process, five core components of CBME curricula were identified: outcome competencies, sequenced progression, tailored learning experiences, competency-focused instruction, and programmatic assessment. With some modification in wording, consensus emerged amongst the panel of international experts. CONCLUSIONS: Typically, implementation evaluation relies on the creation of a specific checklist of practices. Given the ongoing evolution and complexity of CBME, this work, however, focused on identifying core components. Consistent with recent developments in program evaluation, where implementation is described as a developmental trajectory toward fidelity, identifying core components is presented as a fundamental first step toward gaining a more sophisticated understanding of implementation.

Delivering on the promise of competency based medical education - an institutional approach.

The Royal College of Physicians and Surgeons of Canada (RCPSC) adopted a plan to transform, over a seven-year horizon (2014-2021), residency education across all specialties to competency-based medical education (CBME) curriculum models. The RCPSC plan recommended implementing a more responsive and accountable training model with four discrete stages of training, explicit, specialty specific entrustable professional activities, with associated milestones, and a programmatic approach to assessment across residency education. Embracing this vision, the leadership at Queen's University (in Kingston, Ontario, Canada) applied for and was granted special permission by the RCPSC to embark on an accelerated institutional path. Over a three-year period, Queen's took CBME from concept to reality through the development and implementation of a comprehensive strategic plan. This perspective paper describes Queen's University's approach of creating a shared institutional vision, outlines the process of developing a centralized CBME executive team and twenty-nine CBME program teams, and summarizes proactive measures to ensure program readiness for launch. In so doing, Queen's created a community of support and CBME expertise that reinforces shared values including fostering co-production, cultivating responsive leadership, emphasizing diffusion of innovation, and adopting a systems-based approach to transformative change.

A new way to look at simulation-based assessment: the relationship between gaze-tracking and exam performance.

OBJECTIVE: A key task of the team leader in a medical emergency is effective information gathering. Studying information gathering patterns is readily accomplished with the use of gaze-tracking glasses. This technology was used to generate hypotheses about the relationship between performance scores and expert-hypothesized visual areas of interest in residents across scenarios in simulated medical resuscitation examinations. METHODS: Emergency medicine residents wore gaze-tracking glasses during two simulation-based examinations (n=29 and 13 respectively). Blinded experts assessed video-recorded performances using a simulation performance assessment tool that has validity evidence in this context. The relationships between gaze patterns and performance scores were analyzed and potential hypotheses generated. Four scenarios were assessed in this study: diabetic ketoacidosis, bradycardia secondary to beta-blocker overdose, ruptured abdominal aortic aneurysm and metabolic acidosis caused by antifreeze ingestion. RESULTS: Specific gaze patterns were correlated with objective performance. High performers were more likely to fixate on task-relevant stimuli and appropriately ignore task-irrelevant stimuli compared with lower performers. For example, shorter latency to fixation on the vital signs in a case of diabetic ketoacidosis was positively correlated with performance (r=0.70, p<0.05). Conversely, total time spent fixating on lab values in a case of ruptured abdominal aortic aneurysm was negatively correlated with performance (r= −0.50, p<0.05). CONCLUSIONS: There are differences between the visual patterns of high and low-performing residents. These findings may allow for better characterization of expertise development in resuscitation medicine and provide a framework for future study of visual behaviours in resuscitation cases.

The development of national entrustable professional activities to inform the training and assessment of public health and preventative medicine residents.

BACKGROUND: Entrustable Professional Activities (EPAs) have emerged to bridge the gap between the learning of individual competencies and competence in real world practice. EPAs capture the critical core work of a discipline integrating competencies from multiple domains. This report describes the development of a set of EPAs for specialty training in Public Health and Preventive Medicine (PHPM) in Canada. METHODS: The PHPM EPAs were developed using multiple existing sources. A combination of workshops and a national online survey was used to consult with PHPM program directors, the national specialty committee, and competency-based education experts. RESULTS: A national survey of PHPM program directors had a 71% response rate with 80% or more of respondents agreeing with all of the 20 EPA titles and all but one of their descriptions. Competency developmental stage-specific milestones were identified for each EPA. CONCLUSION: The identification of the EPAs and their milestones will increase emphasis on the demonstrated performance of the specialty's core work. Simulations applicable to several EPAs have been developed. The EPAs have also been incorporated into a PHPM National Review Course and will be used to develop a national PHPM curriculum, as well as a national written practice examination.

Evaluating an undergraduate interprofessional simulation-based educational module: communication, teamwork, and confidence performing cardiac resuscitation skills.

PURPOSE: Interprofessional (IP) collaboration during cardiac resuscitation is essential and contributes to patient wellbeing. The purpose of this study is to evaluate an innovative simulation-based IP educational module for undergraduate nursing and medical students on cardiac resuscitation skills. METHODS: Nursing and medical trainees participated in a new cardiac resuscitation curriculum involving a 2-hour IP foundational cardiac resuscitation skills lab, followed by three 2-hour IP simulation sessions. Control group participants attended the existing two 2-hour IP simulation sessions. Study respondents (N = 71) completed a survey regarding their confidence performing cardiac resuscitation skills and their perceptions of IP collaboration. RESULTS: Despite a consistent positive trend, only one out of 17 quantitative survey items were significantly improved for learners in the new curriculum. They were more likely to report feeling confident managing the airway during cardiac resuscitation (P = 0.001). Overall, quantitative results suggest that senior nursing and medical students were comfortable with IP communication and teamwork and confident with cardiac resuscitation skills. There were no significant differences between nursing students' and medical students' results. Through qualitative feedback, participants reported feeling comfortable learning with students from other professions and found value in the IP simulation sessions. CONCLUSION: Results from this study will inform ongoing restructuring of the IP cardiac resuscitation skills simulation module as defined by the action research process. Specific improvements that are suggested by these findings include strengthening the team leader component of the resuscitation skills lab and identifying learners who may benefit from additional practice in the role of team leader and with other skills where they lack confidence.

Assessment of the palatability of vehicles for activated charcoal in pediatric volunteers.

OBJECTIVE: To evaluate the palatability of 4 common flavoring vehicles (water, chocolate milk [CM], orange juice [OJ], and cola) combined with activated charcoal (AC) in pediatric volunteers. DESIGN: A single-blind taste test of 4 different vehicles (water, OJ, a cola drink, and CM) was conducted in healthy volunteer children. Each child tasted 1.25 mL of Charcodote (0.2g/mL) mixed with 1.25 mL of each vehicle. SETTING: Palatability testing was conducted at the office of 1 of the authors. PARTICIPANTS: Thirty children (16 male, 14 female), aged 6.5 +/- 1.4 years (range 5-9 years). OUTCOME MEASURES: After each test dose, each child rated its taste on a modified 10 cm visual analog scale incorporating a facial-hedonic scale. Each child was also asked for his/her overall preference. RESULTS: Taste scores (cm) were as follows: water 5.6 +/- 1.8, OJ 5.4 +/- 1.0, cola 7.6 +/- 0.7, and CM 5.6 +/- 0.8. There was a significant difference in the taste scores between the cola drink (P = 0.01) and the other 3 vehicles. The cola drink was also selected as the most preferred vehicle by 50% of the children as compared with 19.2% for CM and 15.4% for OJ. In contrast, water was selected as the least preferred vehicle by 36.4% of children versus 31.8% for CM and 27.3% for OJ. Only 4 children (15.4%) stated that water was their preferred vehicle, and only 1 child (4.5%) stated that cola drink was the least preferred drink. CONCLUSIONS: Children rate the palatability higher and prefer charcoal given with a cola drink rather than with water. OJ and CM do not seem to improve the acceptability of charcoal.