Evaluating Competence by Design as a Large System Change Initiative: Readiness, Fidelity, and Outcomes.

Program evaluation is an essential, but often neglected, activity in any transformational educational change. Competence by Design was a large-scale change initiative to implement a competency-based time-variable educational system in Canadian postgraduate medical education. A program evaluation strategy was an integral part of the build and implementation plan for CBD from the beginning, providing insights into implementation progress, challenges, unexpected outcomes, and impact. The Competence by Design program evaluation strategy was built upon a logic model and three pillars of evaluation: readiness to implement, fidelity and integrity of implementation, and outcomes of implementation. The program evaluation strategy harvested from both internally driven studies and those performed by partners and invested others. A dashboard for the program evaluation strategy was created to transparently display a real-time view of Competence by Design implementation and facilitate continuous adaptation and improvement. The findings of the program evaluation for Competence by Design drove changes to all aspects of the Competence by Design implementation, aided engagement of partners, supported change management, and deepened our understanding of the journey required for transformational educational change in a complex national postgraduate medical education system. The program evaluation strategy for Competence by Design provides a framework for program evaluation for any large-scale change in health professions education.

Ready, set, go! Evaluating readiness to implement competency-based medical education.

PURPOSE: Organizational readiness is critical for successful implementation of an innovation. We evaluated program readiness to implement Competence by Design (CBD), a model of Competency-Based Medical Education (CBME), among Canadian postgraduate training programs. METHODS: A survey of program directors was distributed 1 month prior to CBD implementation in 2019. Questions were informed by the R = MC2 framework of organizational readiness and addressed: program motivation, general capacity for change, and innovation-specific capacity. An overall readiness score was calculated. An ANOVA was conducted to compare overall readiness between disciplines. RESULTS: Survey response rate was 42% (n = 79). The mean overall readiness score was 74% (30-98%). There was no difference in scores between disciplines. The majority of respondents agreed that successful implementation of CBD was a priority (74%), and that their leadership (94%) and faculty and residents (87%) were supportive of change. Fewer perceived that CBD was a move in the right direction (58%) and that implementation was a manageable change (53%). Curriculum mapping, competence committees and programmatic assessment activities were completed by >90% of programs, while <50% had engaged off-service disciplines. CONCLUSION: Our study highlights important areas where programs excelled in their preparation for CBD, as well as common challenges that serve as targets for future intervention to improve program readiness for CBD implementation.

Training disrupted: Practical tips for supporting competency-based medical education during the COVID-19 pandemic.

The COVID-19 pandemic has disrupted healthcare systems around the world, impacting how we deliver medical education. The normal day-to-day routines have been altered for a number of reasons, including changes to scheduled training rotations, physical distancing requirements, trainee redeployment, and heightened level of concern. Medical educators will likely need to adapt their programs to maximize learning, maintain effective care delivery, and ensure competent graduates. Along with a continued focus on learner/faculty wellness, medical educators will have to optimize existing training experiences, adapt those that are no longer viable, employ new technologies, and be flexible when assessing competencies. These practical tips offer guidance on how to adapt medical education programs within the constraints of the pandemic landscape, stressing the need for communication, innovation, collaboration, flexibility, and planning within the era of competency-based medical education.

Clinician investigator training in Canada: a review.

PURPOSE: The Royal College of Physicians and Surgeons of Canada undertook a review of its Clinician Investigator Program (CIP), 13 years after launching the program in response to shortages in clinical investigators. The primary study goals were to determine the outcomes, impact, strengths and weaknesses of CIP. METHODS: Focus groups and telephone interviews with current and past program directors (PD) and a detailed survey of current and former trainees were conducted. Thirteen PD and 45% of current and former trainees from 10 CIP participated. RESULTS: Since 1995, 12 CIP have been accredited and 553 residents have enrolled in CIP, with 194 completing CIP and residency training by 2008. PD recognized CIP as an excellent program that produces highly qualified clinical investigators; important for faculty renewal. Both trainees and PD identified the need to improve CIP funding. Most (84%) CIP trainees did not have prior graduate degrees. Most alumni had completed Masters (58%) or Doctoral (39%) programs during CIP and published on their CIP research (97%). Among alumni who completed CIP and residency, many obtained an academic appointment with protected time for research, with 39% receiving an external career award. Many (60%) alumni reported no drawbacks to CIP and recognized the added values included Royal College recognition, structured training, pursuit of graduate studies, integration of clinical/research training and enhanced mentorship. CONCLUSION: Since the progam's inception, the number of CIP in Canada has grown. CIP are recognized as important mechanisms for integrating clinical and research training during residency to produce highly qualified clinician investigators.