It's a 'two-way street': resident perspectives of effective coaching relationships in the clinical learning environment.

Background: Coaching has gained traction in postgraduate medical education to enhance interactions between residents and clinical teachers, but these relationships present unique challenges and tensions. In order to realize the promises of coaching in medical education, we must understand how coaching relationships can be enacted to optimize resident development. The purpose of this study was to explore residents' perceptions of key characteristics for effective clinical teacher-resident (CT-R) coaching relationships. Methods: We conducted four focus groups and eight interviews with residents at a Canadian academic center. Using a social constructionist approach, focus groups and interviews were audio recorded, transcribed verbatim, and thematically analyzed. Results: Residents described three main characteristics that contributed to effective CT-R coaching relationships: safe, meaningful, and collaborative. Residents emphasized that these characteristics needed to be bidirectional in nature to be most effective, in that both the resident and clinical teacher embodied these characteristics. Conclusions: Residents identified that effective coaching relationships were shaped not only by clinical teacher behaviours, but importantly, the quality of the interpersonal relationship that was fostered. Thus, it is imperative to consider the bidirectional nature of the CT-R coaching relationship when striving to enhance resident development.

Contexte: Bien que le coaching ait gagné du terrain en éducation médicale postgraduée, notamment comme moyen de renforcer les interactions entre les résidents et les cliniciens enseignants, les rapports entre eux présentent néanmoins des défis et des tensions uniques. Pour atteindre les résultats souhaités en lien avec le coaching en éducation médicale, il faut comprendre comment ces interactions peuvent être mises à profit de façon à optimiser le perfectionnement des résidents. L'objectif de cette étude était d'explorer les perceptions des résidents quant aux caractéristiques clés d'une relation de coaching efficace entre le clinicien enseignant et le résident. Méthodes: Nous avons organisé quatre groupes de discussion et huit entretiens avec des résidents d'un centre universitaire canadien. Suivant une approche socioconstructiviste, les groupes de discussion et les entrevues ont été enregistrés, les verbatims ont été transcrits et ensuite analysés de façon thématique. Résultats: Les résidents ont décrit trois caractéristiques principales de la relation de coaching pouvant contribuer à son efficacité : elle doit être sécuritaire, significative et collaborative. Ils ont insisté sur le fait que pour atteindre une efficacité optimale des interactions de coaching, ces caractéristiques devaient être bidirectionnelles, à savoir présentes tant chez le résident que le clinicien enseignant. Conclusions: D'après les résidents, si l'efficacité de la relation de coaching est déterminée par les comportements des cliniciens enseignants, elle dépend surtout de la qualité des rapports interpersonnels cultivés. Il est donc impératif de tenir compte de la nature bidirectionnelle de la relation de coaching entre cliniciens enseignants et résidents lorsqu'on vise à améliorer le développement de ces derniers.

Operationalizing Programmatic Assessment: The CBME Programmatic Assessment Practice Guidelines.

PROBLEM: Assessing the development and achievement of competence requires multiple formative and summative assessment strategies and the coordinated efforts of trainees and faculty (who often serve in multiple roles, such as academic advisors, program directors, and competency committee members). Operationalizing programmatic assessment (PA) in competency-based medical education (CBME) requires comprehensive practice guidelines, written in accessible language with descriptions of stakeholder activities, to move assessment theory into practice and to help guide the trainees and faculty who enact PA. APPROACH: Informed by the Appraisal of Guidelines for Research and Evaluation II (AGREE II) framework, the authors used a multiphase, multimethod approach to develop the CBME Programmatic Assessment Practice Guidelines (PA Guidelines). The 9 guidelines are organized by phases of assessment and include descriptions of stakeholder activities. A user guide provides a glossary of key terms and summarizes how the guidelines can be used by different stakeholder groups across postgraduate medical education (PGME) contexts. The 4 phases of guideline development, including internal stakeholder consultations and external expert review, occurred between August 2016 and March 2020. OUTCOMES: Local stakeholders and external experts agreed that the PA Guidelines hold potential for guiding initial operationalization and ongoing refinement of PA in CBME by individual stakeholders, residency programs, and PGME institutions. Since July 2020, the PA Guidelines have been used at Queen's University to inform faculty and resident development initiatives, including online CBME modules for faculty, workshops for academic advisors/competence committee members, and a guide that supports incoming residents' transition to CBME. NEXT STEPS: Research exploring the use of the PA Guidelines and user guide in multiple programs and institutions will gather further evidence of their acceptability and utility for guiding operationalization of PA in different contexts.

Training disrupted: Practical tips for supporting competency-based medical education during the COVID-19 pandemic.

The COVID-19 pandemic has disrupted healthcare systems around the world, impacting how we deliver medical education. The normal day-to-day routines have been altered for a number of reasons, including changes to scheduled training rotations, physical distancing requirements, trainee redeployment, and heightened level of concern. Medical educators will likely need to adapt their programs to maximize learning, maintain effective care delivery, and ensure competent graduates. Along with a continued focus on learner/faculty wellness, medical educators will have to optimize existing training experiences, adapt those that are no longer viable, employ new technologies, and be flexible when assessing competencies. These practical tips offer guidance on how to adapt medical education programs within the constraints of the pandemic landscape, stressing the need for communication, innovation, collaboration, flexibility, and planning within the era of competency-based medical education.

Ten ways to get a grip on resident co-production within medical education change.

The Royal College of Physicians and Surgeons of Canada (RCPSC) is transforming its national approach to postgraduate medical education by transitioning all specialty programs to competency based medical education (CBME) curriculums over a seven-year period. Queen's University, with special permission from the RCPSC, launched CBME curricula for all incoming residents across its 29 specialty programs in July 2017. Resident engagement, empowerment, and co-production through this transition has been instrumental in successful implementation of CBME at Queen's University. This article aims to use our own experience at Queen's in the context of current literature and rooted in change leadership theory, to provide a guide for educators, learners, and institutions on how to leverage the interest and enthusiasm of trainees in the transition to CBME in postgraduate training. The following ten tips provides a model for avoiding the "black ice" type pitfalls that can arise with learner involvement and ensure a smoother transition for other institutions moving forward with CBME implementation.

Le Collège royal des médecins et chirurgiens du Canada (CRMCC) s'emploie à transformer son approche nationale à la formation médicale postdoctorale en effectuant une transition, répartie sur une période de sept ans, de tous les programmes spécialisés vers des programmes de formation médicale axéesur les compétences (FMAC). En juillet 2017, l'Université Queen, avec une permission spéciale du CRMCC, a lancé des cursus de FMAC pour tous les nouveaux résidents de ses 29 programmes spécialisés. La participation, la capacité d'agiret la coproduction par lesrésidents pendant cette transition ont contribué à la mise en œuvre réussie de la FMAC à l'Université Queen. Le présent article vise à utiliser notre propre expérience à l'Université Queen dans le contexte de la littérature actuelle et est ancrésur la théorie du leadership en matière de changement, pour procurer un guide aux éducateurs, aux apprenants et aux établissements sur la manière de tirer parti de l'intérêt et de l'enthousiasme des apprenantsdans la transition vers la FMAC dans la formation postdoctorale. Les dix conseils suivants proposent un modèle pour éviter les écueils du type « glace noire ¼ qui peuvent survenir avec la participation de l'apprenant et s'assurer une transition plus harmonieuse pour les autres établissements qui vont de l'avant avec la mise en œuvre de la FMFC.

Competency-based education calls for programmatic assessment: But what does this look like in practice?

RATIONALE, AIMS, AND OBJECTIVES: Programmatic assessment has been identified as a system-oriented approach to achieving the multiple purposes for assessment within Competency-Based Medical Education (CBME, i.e., formative, summative, and program improvement). While there are well-established principles for designing and evaluating programs of assessment, few studies illustrate and critically interpret, what a system of programmatic assessment looks like in practice. This study aims to use systems thinking and the 'two communities' metaphor to interpret a model of programmatic assessment and to identify challenges and opportunities with operationalization. METHOD: An interpretive case study was used to investigate how programmatic assessment is being operationalized within one competency-based residency program at a Canadian university. Qualitative data were collected from residents, faculty, and program leadership via semi-structured group and individual interviews conducted at nine months post-CBME implementation. Data were analyzed using a combination of data-based inductive analysis and theory-derived deductive analysis. RESULTS: In this model, Academic Advisors had a central role in brokering assessment data between communities responsible for producing and using residents' performance information for decision making (i.e., formative, summative/evaluative, and program improvement). As system intermediaries, Academic Advisors were in a privileged position to see how the parts of the assessment system contributed to the functioning of the whole and could identify which system components were not functioning as intended. Challenges were identified with the documentation of residents' performance information (i.e., system inputs); use of low-stakes formative assessments to inform high-stakes evaluative judgments about the achievement of competence standards; and gaps in feedback mechanisms for closing learning loops. CONCLUSIONS: The findings of this research suggest that program stakeholders can benefit from a systems perspective regarding how their assessment practices contribute to the efficacy of the system as a whole. Academic Advisors are well positioned to support educational development efforts focused on overcoming challenges with operationalizing programmatic assessment.

It's a Marathon, Not a Sprint: Rapid Evaluation of Competency-Based Medical Education Program Implementation.

PURPOSE: Despite the broad endorsement of competency-based medical education (CBME), myriad difficulties have arisen in program implementation. The authors sought to evaluate the fidelity of implementation and identify early outcomes of CBME implementation using Rapid Evaluation to facilitate transformative change. METHOD: Case-study methodology was used to explore the lived experience of implementing CBME in the emergency medicine postgraduate program at Queen's University, Canada, using iterative cycles of Rapid Evaluation in 2017-2018. After the intended implementation was explicitly described, stakeholder focus groups and interviews were conducted at 3 and 9 months post-implementation to evaluate the fidelity of implementation and early outcomes. Analyses were abductive, using the CBME core components framework and data-driven approaches to understand stakeholders' experiences. RESULTS: In comparing planned with enacted implementation, important themes emerged with resultant opportunities for adaption. For example, lack of a shared mental model resulted in frontline difficulty with assessment and feedback and a concern that the granularity of competency-focused assessment may result in "missing the forest for the trees," prompting the return of global assessment. Resident engagement in personal learning plans was not uniformly adopted, and learning experiences tailored to residents' needs were slow to follow. CONCLUSIONS: Rapid Evaluation provided critical insights into the successes and challenges of operationalizing CBME. Implementing the practical components of CBME was perceived as a sprint, while realizing the principles of CBME and changing culture in postgraduate training was a marathon requiring sustained effort in the form of frequent evaluation and continuous faculty and resident development.

Simulation versus real-world performance: a direct comparison of emergency medicine resident resuscitation entrustment scoring.

BACKGROUND: Simulation is increasingly being used in postgraduate medical education as an opportunity for competency assessment. However, there is limited direct evidence that supports performance in the simulation lab as a surrogate of workplace-based clinical performance for non-procedural tasks such as resuscitation in the emergency department (ED). We sought to directly compare entrustment scoring of resident performance in the simulation environment to clinical performance in the ED. METHODS: The resuscitation assessment tool (RAT) was derived from the previously implemented and studied Queen's simulation assessment tool (QSAT) via a modified expert review process. The RAT uses an anchored global assessment scale to generate an entrustment score and narrative comments. Emergency medicine (EM) residents were assessed using the RAT on cases in simulation-based examinations and in the ED during resuscitation cases from July 2016 to June 2017. Resident mean entrustment scores were compared using Pearson's correlation coefficient to determine the relationship between entrustment in simulation cases and in the ED. Inductive thematic analysis of written commentary was conducted to compare workplace-based with simulation-based feedback. RESULTS: There was a moderate, positive correlation found between mean entrustment scores in the simulated and workplace-based settings, which was statistically significant (r = 0.630, n = 17, p < 0.01). Further, qualitative analysis demonstrated overall management and leadership themes were more common narratives in the workplace, while more specific task-based feedback predominated in the simulation-based assessment. Both workplace-based and simulation-based narratives frequently commented on communication skills. CONCLUSIONS: In this single-center study with a limited sample size, assessment of residents using entrustment scoring in simulation settings was demonstrated to have a moderate positive correlation with assessment of resuscitation competence in the workplace. This study suggests that resuscitation performance in simulation settings may be an indicator of competence in the clinical setting. However, multiple factors contribute to this complicated and imperfect relationship. It is imperative to consider narrative comments in supporting the rationale for numerical entrustment scores in both settings and to include both simulation and workplace-based assessment in high-stakes decisions of progression.

Impact of night shifts on emergency medicine resident resuscitation performance.

AIM: Emergency medicine (EM) trainees often work nightshifts. We sought to measure how this circadian disruption affects EM resident performance during simulated resuscitations. METHODS: This retrospective cohort study enrolled EM residents at a single Canadian academic centre over a six-year period. Residents completed twice-annual simulation-based resuscitation-focused objective structured clinical examinations (OSCEs) with assessment in four domains (primary assessment, diagnostic actions, therapeutic actions and communication), and a global assessment score (GAS). Primary and secondary exposures of interest were the presence of a nightshift (late-evening shifts ending between midnight and 03h00 or overnight shifts ending after 06h00) the day before or within three days before an OSCE. A random effects linear regression model was used to quantify the association between nightshifts and OSCE scores. RESULTS: From 57 residents, 136 OSCE scores were collected. Working a nightshift the day before an OSCE did not affect male trainee scores but was associated with a significant absolute decrease in mean total scores (-6% [95% CI -12% to 0%]), GAS (-7% [-13% to 0%]), and communication (-9% [-16% to -2%]) scores among women. Working any nightshift within three days before an OSCE lowered absolute mean total scores by 4% [-7% to 0%] and communication scores by 5% [-5% to 0%] irrespective of gender. CONCLUSION: Our results suggest that shift work may impact EM resident resuscitation performance, particularly in the communication domain. This impact may be more significant in women than men, suggesting a need for further investigation.

Competency-based simulation assessment of resuscitation skills in emergency medicine postgraduate trainees - a Canadian multi-centred study.

BACKGROUND: The use of high-fidelity simulation is emerging as a desirable method for competency-based assessment in postgraduate medical education. We aimed to demonstrate the feasibility and validity of a multi-centre simulation-based Objective Structured Clinical Examination (OSCE) of resuscitation competence with Canadian Emergency Medicine (EM) trainees. METHOD: EM postgraduate trainees (n=98) from five Canadian academic centres participated in a high fidelity, 3-station simulation-based OSCE. Expert panels of three emergency physicians evaluated trainee performances at each centre using the Queen's Simulation Assessment Tool (QSAT). Intraclass correlation coefficients were used to measure the inter-rater reliability, and analysis of variance was used to measure the discriminatory validity of each scenario. A fully crossed generalizability study was also conducted for each examination centre. RESULTS: Inter-rater reliability in four of the five centres was strong with a median absolute intraclass correlation coefficient (ICC) across centres and scenarios of 0.89 [0.65-0.97]. Discriminatory validity was also strong (p < 0.001 for scenarios 1 and 3; p < 0.05 for scenario 2). Generalizability studies found significant variations at two of the study centres. CONCLUSIONS: This study demonstrates the successful pilot administration of a multi-centre, 3-station simulation-based OSCE for the assessment of resuscitation competence in post-graduate Emergency Medicine trainees.