Taking stock of what is known about faculty development in competency-based medical education: A scoping review paper.

Purpose: The primary objective was to inventory what is currently known about faculty development (FD) for competency-based medical educations (CBME) and identify gaps in the literature.Methods: A scoping review methodology was employed. Inclusion criteria for article selection were established with two reviewers completing a full-text analysis. Quality checks were included, along with iterative consultation on data collection and consensus decision making via a grounded theory approach.Results: The review identified 19 articles published between 2009 and 2018. Most articles (N = 15) offered suggestions as to what should happen with FD in CBME, but few (N = 4) adopted an experimental design. Six main themes were identified with three main features of FD noted across themes: (1) The importance of direct and timely feedback to faculty members on their teaching and assessment skills. (2) The role of establishing shared mental models for CBME curricula. (3) That FD is thought of longitudinally, not as a one-time bolus.Conclusion: This work illustrates that there is limited, high quality research in FD for CBME. Future FD activities should consider employing a longitudinal and multi-modal program format that includes feedback for the faculty participants on their teaching and assessments skills, including the development of faculty coaching skills.

Managing residents in difficulty within CBME residency educational systems: a scoping review.

BACKGROUND: Best practices in managing residents in difficulty (RID) in the era of competency-based medical education (CBME) are not well described. This scoping review aimed to inventory the current literature and identify major themes in the articles that address or employ CBME as part of the identification and remediation of residents in difficulty. METHODS: Articles published between 2011 to 2017 were included if they were about postgraduate medical education, RID, and offered information to inform the structure and/or processes of CBME. All three reviewers performed a primary screening, followed by a secondary screening of abstracts of the chosen articles, and then a final comprehensive sub-analysis of the 11 articles identified as using a CBME framework. RESULTS: Of 165 articles initially identified, 92 qualified for secondary screening; the 63 remaining articles underwent full-text abstracting. Ten themes were identified from the content analysis with "identification of RID" (41%) and "defining and classifying deficiencies" (30%) being the most frequent. In the CBME article sub-analysis, the most frequent themes were: need to identify RID (64%), improving assessment tools (45%), and roles and responsibilities of players involved in remediation (27%). Almost half of the CBME articles were published in 2016-2017. CONCLUSIONS: Although CBME programs have been implemented for many years, articles have only recently begun specifically addressing RID within a competency framework. Much work is needed to describe the sequenced progression, tailored learning experiences, and competency-focused instruction. Finally, future research should focus on the outcomes of remediation in CBME programs.

Twelve tips for developing key-feature questions (KFQ) for effective assessment of clinical reasoning.

Clinical reasoning is the cognitive process that makes it possible for us to reach conclusions from clinical data. "A key feature (KF) is defined as a significant step in the resolution of a clinical problem. Examinations using key-feature questions (KFQs) focus on a challenging aspect in the diagnosis and management of a clinical problem where the candidates are most likely to make errors." KFs have been used at different levels of medical education and practice, from undergraduate to certification examinations. KFQs illuminate the strengths and limits of an individual's clinical problem-solving ability. These types of items are more likely than other forms of assessment to discriminate among stronger or weaker candidates in the area of clinical reasoning. The 12 tips in this article will provide guidance to faculty who wish to develop KFQs for their tests.

Eight-year outcomes of a competency-based residency training program in orthopedic surgery.

Background: The Division of Orthopaedic Surgery at the University of Toronto implemented a pilot residency training program that used a competency-based framework in July of 2009. The competency-based curriculum (CBC) deployed an innovative, modularized approach that dramatically intensified both the structured learning elements and the assessment processes. Methods: This paper discusses the initial curriculum design of the CBC pilot program; the refinement of the curriculum using curriculum mapping that allowed for efficiencies in educational delivery; details of evaluating resident competence; feedback from external reviews by accrediting bodies; and trainee and program outcomes for the first eight years of the program's implementation. Results: Feedback from the residents, the faculty, and the postgraduate residency training accreditation bodies on the CBC has been positive and suggests that the essential framework of the program may provide a valuable tool to other programs that are contemplating embarking on transition to competency-based education. Conclusions: While the goal of the program was not to shorten training per se, efficiencies gained through a modular, competency-based program have resulted in shortened time to completion of residency training for some learners.

What do I do? Developing a competency inventory for postgraduate (residency) program directors.

BACKGROUND: Few new Residency Program Directors (PD) are formally trained for the demands and responsibilities of the leadership aspect of their role. Currently, there are no comprehensive frameworks that describe specific leadership competencies that can inform PD self-reflection or faculty development. METHODS: The authors developed a Postgraduate Program Director Competency Inventory (PPDCI) in order to frame the performance of PDs for a multisource feedback (MSF) program. The development of the PPDCI occurred in five phases which involved: development of an initial inventory, implementation of a key informant survey of national opinion leaders, execution of a validity survey with postgraduate education leaders and committee members and implementation of a further refined inventory with 17 PD and 147 raters as part of a pilot MSF program. OUTCOMES: Five distinct domains of leadership competence were identified which included: Communication and relationship management, leadership, professionalism and self-management, environmental engagement, and management skills and knowledge. The content validity of the PPDCI was endorsed by 85% of the key informants. The validity survey indicated strong endorsement of the PPDCI domains and recognition of its utility for both orientation of new PD as well as a frame for self-assessment. The pilot MSF program yielded a further refined and reduced inventory of 26 items of competence as well as recommendations for its utility. CONCLUSIONS: Use of this leadership inventory has the potential to ensure effective leadership of postgraduate programs.

How to assess communication, professionalism, collaboration and the other intrinsic CanMEDS roles in orthopedic residents: use of an objective structured clinical examination (OSCE).

BACKGROUND: Assessing residents' understanding and application of the 6 intrinsic CanMEDS roles (communicator, professional, manager, collaborator, health advocate, scholar) is challenging for postgraduate medical educators. We hypothesized that an objective structured clinical examination (OSCE) designed to assess multiple intrinsic CanMEDS roles would be sufficiently reliable and valid. METHODS: The OSCE comprised 6 10-minute stations, each testing 2 intrinsic roles using case-based scenarios (with or without the use of standardized patients). Residents were evaluated using 5-point scales and an overall performance rating at each station. Concurrent validity was sought by correlation with in-training evaluation reports (ITERs) from the last 12 months and an ordinal ranking created by program directors (PDs). RESULTS: Twenty-five residents from postgraduate years (PGY) 0, 3 and 5 participated. The interstation reliability for total test scores (percent) was 0.87, while reliability for each of the communicator, collaborator, manager and professional roles was greater than 0.8. Total test scores, individual station scores and individual CanMEDS role scores all showed a significant effect by PGY level. Analysis of the PD rankings of intrinsic roles demonstrated a high correlation with the OSCE role scores. A correlation was seen between ITER and OSCE for the communicator role, while the ITER medical expert and total scores highly correlated with the communicator, manager and professional OSCE scores. CONCLUSION: An OSCE designed to assess the intrinsic CanMEDS roles was sufficiently valid and reliable for regular use in an orthopedic residency program.

CONTEXTE: Évaluer la compréhension et l'application des 6 rôles intrinsèques CanMEDS (communicateur, professionnel, gestionnaire, collaborateur, promoteur de la santé, érudit) chez les résidents pose un défi pour les responsables de la formation médicale postdoctorale. Nous avons émis l'hypothèse selon laquelle un examen clinique objectif structuré (ECOS) conçu pour évaluer plusieurs rôles CanMEDS intrinsèques serait suffisamment fiable et valide. MÉTHODES: L'ECOS comportait 6 stations de 10 minutes, permettant chacune d'évaluer 2 rôles intrinsèques à l'aide de scénarios basés sur des cas (avec ou sans recours à des patients standardisés). Les résidents ont été notés au moyen d'échelles en 5 points et d'une évaluation globale de leur rendement à chacune des stations. La validité convergente a été vérifiée par corrélation avec les rapports d'évaluation en cours de formation (RÉF) des 12 mois précédents et un classement chiffré créé par les directeurs du programme (DP). RÉSULTATS: Vingt-cinq résidents des années 0, 3 et 5 y ont participé. La fiabilité interstation pour les scores totaux aux tests (en pourcentage) a été de 0,87, tandis que la fiabilité pour chacun des rôles de communicateur, collaborateur, gestionnaire et professionnel, a été supérieure à 0,8. Les scores totaux aux tests, les scores aux stations individuelles et les scores pour les rôles CanMEDS individuels ont tous fait état d'un effet significatif selon le niveau des résidents. L'analyse des classements établis par les DP quant aux rôles intrinsèques a révélé une forte corrélation avec les scores au test ECOS pour les rôles. On a observé une corrélation entre les RÉF et l'ECOS pour le rôle de communicateur, tandis que les RÉF pour le rôle d'expert médical et les scores totaux ont été en forte corrélation avec les scores de l'ECOS pour les rôles de communicateur, de gestionnaire et de professionnel. CONCLUSION: Un ECOS conçu pour évaluer les rôles CanMEDS intrinsèques s'est révélé suffisamment valide et fiable pour un usage régulier dans un programme de résidence en orthopédie.

Collective Wisdom in Faculty Development for Competency-Based Medical Education: A Needs Assessment and Survey.

This article was migrated. The article was marked as recommended. Background: As programs transition to competency-based medical education (CBME), faculty development (FD) will be a key component of supporting successful implementation. Methods: Faculty at the University of Toronto (UofT) with leadership roles in residency education were invited to complete an online survey to explore their experiences with FD for CBME. Descriptive statistics were collected. Results were analyzed using thematic, frequency and comparative analyses between respondent subgroups to identify trends and theme categories relevant to the perceived most effective FD activities, most helpful FD topics as well as program/department needs for future FD initiatives. Results: The overall survey response rate was 44.6%. The most effective FD activity identified by survey respondents was a small group format. Perceived top FD topics included implementing CBME, assessment tools, feedback and coaching along with competence committees. The majority of programs felt that the ideal timing for CBME implementation was 6-12 months prior to implementation. The main perceived barrier to FD was lack of time amongst faculty. Conclusions: This data helped drive FD at UofT by supporting strategic planning for implementing competency based curricular reforms. The results have also informed the need for additional resources and enable focused FD on barriers and needs.

Epidemiology of competence: a scoping review to understand the risks and supports to competence of four health professions.

OBJECTIVES: This study examined the risks and supports to competence discussed in the literature related to occupational therapists, pharmacists, physical therapists and physicians, using epidemiology as a conceptual model. DESIGN: Articles from a scoping literature review, published from 1975 to 2014 inclusive, were included if they were about a risk or support to the professional or clinical competence of one of four health professions. Descriptive and regression analyses identified potential associations between risks and supports to competence and the location of study, type of health profession, competence life-cycle and the domain(s) of competence (organised around the CanMEDS framework). RESULTS: A total of 3572 abstracts were reviewed and 943 articles analysed. Most focused on physicians (n=810, 86.0%) and 'practice' (n=642, 68.0%). Fewer articles discussed risks to competence (n=418, 44.3%) than supports (n=750, 79.5%). The top four risks, each discussed in over 15% of articles, were: transitions in practice, being an international graduate, lack of clinical exposure/experience (ie, insufficient volume of procedures or patients) and age. The top two supports (over 35%) were continuing education participation and educational information/programme features. About 60% of all the articles discussed medical expert and about 25% applied to all roles. Articles focusing on residents had a greater probability of reporting on risks. CONCLUSIONS: Articles about physicians were dominant. The majority of articles were written in the last decade and more discussed supports than risks to competence. An epidemiology-based conceptual model offers a helpful organising framework for exploring and explaining the competence of health professions.

The epidemiology of competence: protocol for a scoping review.

INTRODUCTION: Factors that are important to the competence of healthcare providers have important consequences for quality of healthcare. Although some previous research has discussed risks or supports to the competence of clinicians, a thorough exploration is currently lacking. The purpose of this review is to examine the literature examining risks and supports to clinical competence of healthcare practitioners and trainees engaged in field-based education. In this study, field-based education refers to teaching or training in a real-world/work-based setting. METHODS AND ANALYSIS: We will employ an established scoping review methodology. Eligible studies will include those that mention (1) a healthcare professional, (2) competence from field-based education throughout the lifespan/career and (3) a risk or support to competence. Four authors will independently apply the inclusion criteria to all studies, regardless of study design. Our data extraction will include information on study design, location and type of study and we will develop a comprehensive list of risks and supports that are discussed in the literature. ETHICS AND DISSEMINATION: Since this is a review of the literature, ethics approval is not indicated. We will disseminate the findings from this study in publications in peer-reviewed journals as well as presentations at relevant national and international conferences.

Postgraduate anaesthesiology education: protocol for a scoping review.

INTRODUCTION: The link between education and clinical practice is vital, yet the current state of research suggests there is a substantial gap between medical education research and practice. This too is the case in the domain of anaesthesiology education research, as much of the research focuses on simulation studies, and a narrow range of research methods. The aim of this study is to comprehensively review the existing literature in postgraduate anaesthesiology education research in order to identify key research priorities. The findings from this review will be used to establish a base for developing a strategic research programme in anaesthesia education and practice. METHODS AND ANALYSIS: We will employ the scoping review methodology outlined by Arksey and O'Malley (2005) to comprehensively search the literature pertaining to postgraduate anaesthesiology education. We will search relevant electronic databases (eg, MEDLINE, EMBASE) and grey literature. After conducting calibration exercises, two authors will independently apply inclusion criteria to all titles and abstracts and perform full-text review of all eligible articles. Data to be extracted will include basic information about the study (eg, location, design) as well as detailed information regarding the context of the research and type of education examined. Our results will be used to develop a framework of themes that outline the research currently being conducted and identify gaps in research. ETHICS AND DISSEMINATION: This review is the first step in a strategic research plan in postgraduate anaesthesiology education. We plan to disseminate this research through publications, presentations and meetings with relevant stakeholders. Ethical approval was not sought for this scoping review.