Competence By Design: a transformational national model of time-variable competency-based postgraduate medical education.

Postgraduate medical education is an essential societal enterprise that prepares highly skilled physicians for the health workforce. In recent years, PGME systems have been criticized worldwide for problems with variable graduate abilities, concerns about patient safety, and issues with teaching and assessment methods. In response, competency based medical education approaches, with an emphasis on graduate outcomes, have been proposed as the direction for 21st century health profession education. However, there are few published models of large-scale implementation of these approaches. We describe the rationale and design for a national, time-variable competency-based multi-specialty system for postgraduate medical education called Competence by Design. Fourteen innovations were bundled to create this new system, using the Van Melle Core Components of competency based medical education as the basis for the transformation. The successful execution of this transformational training system shows competency based medical education can be implemented at scale. The lessons learned in the early implementation of Competence by Design can inform competency based medical education innovation efforts across professions worldwide.

Competency-Based Medical Education at Scale: A Road Map for Transforming National Systems of Postgraduate Medical Education.

In the past decade, the Canadian system of postgraduate medical education has been transformed with the implementation of a new approach to competency based medical education called Competence by Design. The Royal College of Physicians and Surgeons of Canada (Royal College) developed an approach to time-variable competency based medical education and adapted that design for medical, surgical, and diagnostic disciplines. New educational standards and entrustable professional activities consistent with this approach were co-created with 67 specialties and subspecialties, and implementation was scaled up across 17 universities and over 1000 postgraduate training programs. Partner engagement, systematic design of workshops to create discipline specific competency-based standards of education, and agile adaptation were all key ingredients for success. This paper describes the strategies applied by the Royal College, lessons learned regarding transformative change in the complex system of postgraduate medical education, and the current status of the Competence by Design initiative. The approach taken and lessons learned by the Royal College may be useful for other educators who are planning a transformation to CBME or any other major educational reform.

Enabling Implementation of Competency Based Medical Education through an Outcomes-Focused Accreditation System.

Competency based medical education is being adopted around the world. Accreditation plays a vital role as an enabler in the adoption and implementation of competency based medical education, but little has been published about how the design of an accreditation system facilitates this transformation. The Canadian postgraduate medical education environment has recently transitioned to an outcomes-based accreditation system in parallel with the adoption of competency based medical education. Using the Canadian example, we characterize four features of an accreditation system that can facilitate the implementation of competency based medical education: theoretical underpinning, quality focus, accreditation standards, and accreditation processes. Alignment of the underlying educational theories within the accreditation system and educational paradigm drives change in a consistent and desired direction. An accreditation system that prioritizes quality improvement over quality assurance promotes educational system development and progressive change. Accreditation standards that achieve the difficult balance of being sufficiently detailed yet flexible foster a high fidelity of implementation without stifling innovation. Finally, accreditation processes that recognize the change process, encourage program development, and are not overly punitive all enable the implementation of competency based medical education. We also discuss the ways in which accreditation can simultaneously hinder the implementation of this approach. As education bodies adopt competency based medical education, particular attention should be paid to the role that accreditation plays in successful implementation.

Ready, set, go! Evaluating readiness to implement competency-based medical education.

PURPOSE: Organizational readiness is critical for successful implementation of an innovation. We evaluated program readiness to implement Competence by Design (CBD), a model of Competency-Based Medical Education (CBME), among Canadian postgraduate training programs. METHODS: A survey of program directors was distributed 1 month prior to CBD implementation in 2019. Questions were informed by the R = MC2 framework of organizational readiness and addressed: program motivation, general capacity for change, and innovation-specific capacity. An overall readiness score was calculated. An ANOVA was conducted to compare overall readiness between disciplines. RESULTS: Survey response rate was 42% (n = 79). The mean overall readiness score was 74% (30-98%). There was no difference in scores between disciplines. The majority of respondents agreed that successful implementation of CBD was a priority (74%), and that their leadership (94%) and faculty and residents (87%) were supportive of change. Fewer perceived that CBD was a move in the right direction (58%) and that implementation was a manageable change (53%). Curriculum mapping, competence committees and programmatic assessment activities were completed by >90% of programs, while <50% had engaged off-service disciplines. CONCLUSION: Our study highlights important areas where programs excelled in their preparation for CBD, as well as common challenges that serve as targets for future intervention to improve program readiness for CBD implementation.

Rapid deployment of a virtual simulation curriculum to prepare for critical care triage during the COVID-19 pandemic.

BACKGROUND: During the COVID-19 pandemic in Ontario, Canada, an Emergency Standard of Care for Major Surge was created to establish a uniform process for the "triage" of finite critical care resources. This proposed departure from usual clinical care highlighted the need for an educational tool to prepare physicians for making and communicating difficult triage decisions. We created a just-in-time, virtual, simulation-based curriculum and evaluated its impact for our group of academic Emergency Physicians. METHODS: Our curriculum was developed and evaluated following Stufflebeam's Context-Input-Process-Product model. Our virtual simulation sessions, delivered online using Microsoft Teams, addressed a range of clinical scenarios involving decisions about critical care prioritization (i.e., Triage). Simulation participants completed a pre-course multiple-choice knowledge test and rating scales pertaining to their attitudes about using the Emergency Standard of Care protocol before and 2-4 weeks after participating. Qualitative feedback about the curriculum was solicited through surveys. RESULTS: Nine virtual simulation sessions were delivered over 3 weeks, reaching a total of 47 attending emergency physicians (74% of our active department members). Overall, our intervention led to a 36% (95% CI 22.9-48.3%) improvement in participants' self-rated comfort and attitudes in navigating triage decisions and communicating with patients at the end of life. Scores on the knowledge test improved by 13% (95% CI 0.4-25.6%). 95% of participants provided highly favorable ratings of the course content and similarly indicated that the session was likely or very likely to change their practice. The curriculum has since been adopted at multiple sites around the province. CONCLUSION: Our novel virtual simulation curriculum facilitated rapid dissemination of the Emergency Standard of Care for Major Surge to our group of Emergency Physicians despite COVID-19-related constraints on gathering. The active learning afforded by this method improved physician confidence and knowledge with these difficult protocols.

RéSUMé: CONTEXTE: Au cours de la pandémie de COVID-19 en Ontario, au Canada, une norme de soins d'urgence pour les poussées majeures a été créée afin d'établir un processus uniforme pour le " triage " des ressources limitées en soins intensifs. Cette proposition d'écart par rapport aux soins cliniques habituels a mis en évidence la nécessité d'un outil éducatif pour préparer les médecins à prendre et à communiquer des décisions de triage difficiles. Nous avons créé un programme d'études virtuel, juste à temps, basé sur la simulation et avons évalué son impact sur notre groupe de médecins urgentistes universitaires. MéTHODES: Notre programme d'études a été développé et évalué selon le modèle Contexte-Intrant-Processus-Produit de Stufflebeam. Nos sessions de simulation virtuelle, réalisées en ligne à l'aide de Microsoft Teams, ont abordé une série de scénarios cliniques impliquant des décisions sur la priorisation des soins intensifs (c.-à-d. le triage). Les participants à la simulation ont rempli un test de connaissances à choix multiples avant le cours et des échelles d'évaluation concernant leurs attitudes à l'égard de l'utilisation du protocole de soins d'urgence standard avant et deux à quatre semaines après leur participation. Des commentaires qualitatifs sur le programme ont été sollicités par le biais d'enquêtes. RéSULTATS: Neuf sessions de simulation virtuelle ont été dispensées sur trois semaines, touchant au total 47 médecins urgentistes titulaires (74 % des membres actifs de notre service). Dans l'ensemble, notre intervention a conduit à une amélioration de 36 % (IC 95 % 22,9-48,3 %) de l'auto-évaluation du confort et des attitudes des participants en matière de décisions de triage et de communication avec les patients en fin de vie. Les scores au test de connaissances se sont améliorés de 13% (IC 95% 0,4-25,6%). 95 % des participants ont donné une évaluation très favorable du contenu du cours et ont également indiqué que la session était susceptible ou très susceptible de modifier leur pratique. Le programme d'études a depuis été adopté à plusieurs endroits dans la province. CONCLUSION: Notre nouveau programme de simulation virtuelle a facilité la diffusion rapide des normes de soins d'urgence en cas de crise majeure à notre groupe d'urgentistes, malgré les contraintes de rassemblement liées au COVID-19. L'apprentissage actif que permet cette méthode a amélioré la confiance et les connaissances des médecins concernant ces protocoles difficiles.

Implementation of competence committees during the transition to CBME in Canada: A national fidelity-focused evaluation.

PURPOSE: This study evaluated the fidelity of competence committee (CC) implementation in Canadian postgraduate specialist training programs during the transition to competency-based medical education (CBME). METHODS: A national survey of CC chairs was distributed to all CBME training programs in November 2019. Survey questions were derived from guiding documents published by the Royal College of Physicians and Surgeons of Canada reflecting intended processes and design. RESULTS: Response rate was 39% (113/293) with representation from all eligible disciplines. Committee size ranged from 3 to 20 members, 42% of programs included external members, and 20% included a resident representative. Most programs (72%) reported that a primary review and synthesis of resident assessment data occurs prior to the meeting, with some data reviewed collectively during meetings. When determining entrustable professional activity (EPA) achievement, most programs followed the national specialty guidelines closely with some exceptions (53%). Documented concerns about professionalism, EPA narrative comments, and EPA entrustment scores were most highly weighted when determining resident progress decisions. CONCLUSIONS: Heterogeneity in CC implementation likely reflects local adaptations, but may also explain some of the variable challenges faced by programs during the transition to CBME. Our results offer educational leaders important fidelity data that can help inform the larger evaluation and transformation of CBME.

Outcome Feedback within Emergency Medicine Training Programs: An Opportunity to Apply the Theory of Deliberate Practice?

OBJECTIVES: Outcome feedback is the process of learning patient outcomes after their care within the emergency department. We conducted a national survey of Canadian Royal College emergency medicine (EM) residents and program directors to determine the extent to which active outcome feedback and follow-up occurred. We also compared the perceived educational value of outcome feedback between residents and program directors. METHODS: We distributed surveys to all Royal College-accredited adult and pediatric EM training programs using a modified Dillman method. We analyzed the data using student's t-test for continuous variables and Fisher's exact test for categorical variables. RESULTS: We received 210 completed surveys from 260 eligible residents (80.8%) and 21 of 24 program directors (87.5%) (overall 81.3%). Mandatory active outcome feedback was not present in any EM training program for admitted or discharged patients (0/21). Follow-up was performed electively by 89.4% of residents for patients admitted to the hospital, and by 44.2% of residents for patients discharged home. A majority of residents (76.9%) believed that patient follow-up should be mandatory compared to 42.9% of program directors (p=0.002). The perceived educational value of outcome feedback was 5.8/7 for residents and 5.1/7 for program directors (difference 0.7; p=0.002) based on a seven-point Likert scale (1=not important; 7=very important). CONCLUSION: While Canadian EM training programs do not mandate follow-up, it is performed electively by the majority of residents surveyed. Residents place a significantly greater educational value on outcome feedback than their program directors, and believe that follow-up should be a mandatory component of EM residencies.

Positive perception of pharmacogenetic testing for psychotropic medications.

INTRODUCTION: Pharmacogenetics attempts to identify inter-individual genetic differences that are predictive of variable drug response and propensity to side effects, with the prospect of assisting physicians to select the most appropriate drug and dosage for treatment. However, many concerns regarding genetic tests exist. We sought to test the opinions of undergraduate science and medical students in southern Ontario universities toward pharmacogenetic testing. METHODS AND RESULTS: Questionnaires were completed by 910 undergraduate medicine and science students from 2005 to 2007. Despite students' concerns that the results of genetic tests may be used for other purposes without consent (71%) or lead to discrimination (78%), an overwhelming number of students were in favor of pharmacogenetic testing (90%). DISCUSSION: To our knowledge, this study is the first to survey a large sample for their attitude toward pharmacogenetic testing for psychotropic medications. Our results indicate that, although concerns remain and scientific advancements are required, respondents were in support of pharmacogenetic testing for medications used to treat schizophrenia. © 2014 The Authors. Human Psychopharmacology: Clinical and Experimental published by John Wiley & Sons, Ltd.