Making sense of competency-based medical education (CBME) literary conversations: A BEME scoping review: BEME Guide No. 78.

BACKGROUND: Competency-based medical education (CBME) received increased attention in the early 2000s by educators, clinicians, and policy makers as a way to address concerns about physician preparedness and patient safety in a rapidly changing healthcare environment. Opinions and perspectives around this shift in medical education vary and, to date, a systematic search and synthesis of the literature has yet to be undertaken. The aim of this scoping review is to present a comprehensive map of the literary conversations surrounding CBME. METHODS: Twelve different databases were searched from database inception up until 29 April 2020. Literary conversations were extracted into the following categories: perceived advantages, perceived disadvantages, challenges/uncertainties/skepticism, and recommendations related to CBME. RESULTS: Of the 5757 identified records, 387 were included in this review. Through thematic analysis, eight themes were identified in the literary conversations about CBME: credibility, application, community influence, learner impact, assessment, educational developments, organizational structures, and societal impacts of CBME. Content analysis supported the development of a heat map that provides a visual illustration of the frequency of these literary conversations over time. CONCLUSIONS: This review serves two purposes for the medical education research community. First, this review acts as a comprehensive historical record of the shifting perceptions of CBME as the construct was introduced and adopted by many groups in the medical education global community over time. Second, this review consolidates the many literary conversations about CBME that followed the initial proposal for this approach. These findings can facilitate understanding of CBME for multiple audiences both within and outside of the medical education research community.

Development, Implementation, and Meta-Evaluation of a National Approach to Programmatic Assessment in Canadian Family Medicine Residency Training.

The growing international adoption of competency-based medical education has created a desire for descriptions of innovative assessment approaches that generate appropriate and sufficient information to allow for informed, defensible decisions about learner progress. In this article, the authors provide an overview of the development and implementation of the approach to programmatic assessment in postgraduate family medicine training programs in Canada, called Continuous Reflective Assessment for Training (CRAFT). CRAFT is a principles-guided, high-level approach to workplace-based assessment that was intentionally designed to be adaptable to local contexts, including size of program, resources available, and structural enablers and barriers. CRAFT has been implemented in all 17 Canadian family medicine residency programs, with each program taking advantage of the high-level nature of the CRAFT guidelines to create bespoke assessment processes and tools appropriate for their local contexts. Similarities and differences in CRAFT implementation between 5 different family medicine residency training programs, representing both English- and French-language programs from both Western and Eastern Canada, are described. Despite the intentional flexibility of the CRAFT guidelines, notable similarities in assessment processes and procedures across the 5 programs were seen. A meta-evaluation of findings from programs that have published evaluation information supports the value of CRAFT as an effective approach to programmatic assessment. While CRAFT is currently in place in family medicine residency programs in Canada, given its adaptability to different contexts as well as promising evaluation data, the CRAFT approach shows promise for application in other training environments.

Making sense of adaptive expertise for frontline clinical educators: a scoping review of definitions and strategies.

Adaptive expertise has been promoted as an emerging model of expertise in health professions education in response to the inherent complexities of patient care; however, as the concept increasingly influences the structure of professional training and practice, it creates the potential for misunderstandings of the definition and implications of adaptive expertise. To foster a common understanding of the concept, we conducted a scoping review to explore how adaptive expertise has been discussed within health professions education literature. Five databases-MedLine, PubMed, ERIC, CINAHL, and PsycINFO-were searched using the exact term "adaptive expertise", producing 212 unique articles. Fifty-eight articles met inclusion criteria. In the included articles, authors discussed the conceptual implications of adaptive expertise for health professions education, strategies for training for adaptive expertise, and research findings aimed at supporting the development of adaptive expertise or utilizing adaptive expertise as a theoretical framework. The goal of this scoping review is to establish a resource for frontline educators tasked with fostering the development of adaptive expertise in learners through education initiatives. A common understanding of adaptive expertise is essential to ensuring effective implementation in training programs.

Using Learning Analytics to Examine Differences in Assessment Forms From Continuous Versus Episodic Supervisors of Family Medicine Residents.

Background: It is assumed that there is a need for continuity of supervision within competency-based medical education, despite most evidence coming from the undergraduate medical education rather than the graduate medical education (GME) context. This evidence gap must be addressed to justify the time and effort needed to redesign GME programs to support continuity of supervision. Objective: To examine differences in assessment behaviors of continuous supervisors (CS) versus episodic supervisors (ES), using completed formative assessment forms, FieldNotes, as a proxy. Methods: The FieldNotes CS- and ES-entered for family medicine residents (N=186) across 3 outpatient teaching sites over 3 academic years (2015-2016, 2016-2017, 2017-2018) were examined using 2-sample proportion z-tests to determine differences on 3 FieldNote elements: competency (Sentinel Habit [SH]), Clinical Domain (CD), and Progress Level (PL). Results: Sixty-nine percent (6104 of 8909) of total FieldNotes were analyzed. Higher proportions of CS-entered FieldNotes indicated SH3 (Managing patients with best practices), z=-3.631, P<.0001; CD2 (Care of adults), z=-8.659, P<.0001; CD3 (Care of the elderly), z=-4.592, P<.0001; and PL3 (Carry on, got it), z=-4.482, P<.0001. Higher proportions of ES-entered FieldNotes indicated SH7 (Communication skills), z=4.268, P<.0001; SH8 (Helping others learn), z=20.136, P<.0001; CD1 (Doctor-patient relationship/ethics), z=14.888, P<.0001; CD9 (Not applicable), z=7.180, P<.0001; and PL2 (In progress), z=5.117, P<.0001. Conclusions: The type of supervisory relationship impacts assessment: there is variability in which competencies are paid attention to, which contexts or populations are included, and which progress levels are chosen.

A Comparison of Resident-Completed and Preceptor-Completed Formative Workplace-Based Assessments in a Competency-Based Medical Education Program.

BACKGROUND AND OBJECTIVES: In competency-based medical education (CBME), should resident self-assessments be included in the array of evidence upon which summative progress decisions are made? We examined the congruence between self-assessments and preceptor assessments of residents using assessment data collected in a 2-year Canadian family medicine residency program that uses programmatic assessment as part of their approach to CBME. METHODS: This was a retrospective observational cohort study using a learning analytics approach. The data source was archived formative workplace-based assessment forms (fieldnotes) stored in an online portfolio by family medicine residents and preceptors. Data came from three academic teaching sites over 3 academic years (2015-2016, 2016-2017, 2017-2018), and were analyzed in aggregate using nonparametric tests to evaluate differences in progress levels selected both within and between groups. RESULTS: In aggregate, first-year residents' self-reported progress was consistent with that indicated by preceptors. Progress level rating on fieldnotes improved over training in both groups. Second-year residents tended to assign themselves higher ratings on self-entered assessments compared with those assigned by preceptors; however, the effect sizes associated with these findings were small. CONCLUSIONS: Although we found differences in the progress level selected between preceptor-entered and resident-entered fieldnotes, small effect sizes suggest these differences may have little practical significance. Reasonable consistency between resident self-assessments and preceptor assessments suggests that benefits of guided self-assessment (eg, support of self-regulated learning, program efficacy monitoring) remain appealing despite potential risks.

Development of and Preliminary Validity Evidence for the EFeCT Feedback Scoring Tool.

BACKGROUND: Narrative feedback, like verbal feedback, is essential to learning. Regardless of form, all feedback should be of high quality. This is becoming even more important as programs incorporate narrative feedback into the constellation of evidence used for summative decision-making. Continuously improving the quality of narrative feedback requires tools for evaluating it, and time to score. A tool is needed that does not require clinical educator expertise so scoring can be delegated to others. OBJECTIVE: To develop an evidence-based tool to evaluate the quality of documented feedback that could be reliably used by clinical educators and non-experts. METHODS: Following a literature review to identify elements of high-quality feedback, an expert consensus panel developed the scoring tool. Messick's unified concept of construct validity guided the collection of validity evidence throughout development and piloting (2013-2020). RESULTS: The Evaluation of Feedback Captured Tool (EFeCT) contains 5 categories considered to be essential elements of high-quality feedback. Preliminary validity evidence supports content, substantive, and consequential validity facets. Generalizability evidence supports that EFeCT scores assigned to feedback samples show consistent interrater reliability scores between raters across 5 sessions, regardless of level of medical education or clinical expertise (Session 1: n=3, ICC=0.94; Session 2: n=6, ICC=0.90; Session 3: n=5, ICC=0.91; Session 4: n=6, ICC=0.89; Session 5: n=6, ICC=0.92). CONCLUSIONS: There is preliminary validity evidence for the EFeCT as a useful tool for scoring the quality of documented feedback captured on assessment forms. Generalizability evidence indicated comparable EFeCT scores by raters regardless of level of expertise.

Effective competency-based medical education requires learning environments that promote a mastery goal orientation: A narrative review.

PURPOSE: Competency-based medical education (CBME) emphasizes the need for learners to be central to their own learning and to take an active role in learning. This approach has a dual aim: to encourage learners to actively engage in their own learning, and to push learners to develop learning strategies that will prepare them for lifelong learning. This review paper proposes a theoretical bridge between CBME and lifelong learning and puts forth the argument that in order for CBME programs to produce the physicians truly needed in our society now and in the future, learning environments must be intentionally designed to foster mastery goal orientations and to support the development of adaptive self-regulated learning skills and behaviours. MATERIALS AND METHODS: This narrative literature review incorporated results of searches conducted by a subject librarian in PsycInfo and MedLine. Articles were also identified through reference lists of identified papers to capture older key citations. Analysis of the literature used a constructivist epistemological approach to develop an integrative description of the interaction of achievement goal orientation, self-regulated learning, learning environment, and lifelong learning. RESULTS: Findings from achievement goal theory research support the assumption that adoption of a mastery goal orientation facilitates the use of adaptive learning behaviours, such as those described in self-regulated learning theory. Adaptive self-regulated learning strategies, in turn, facilitate effective lifelong learning. The authors offer evidence for how learning environments influence goal orientations and self-regulated learning, and propose that CBME programs intentionally plan for such learning environments. Finally, the authors offer specific suggestions and examples for how learning environments can be designed or adjusted to support adoption of a mastery goal orientation and use of self-regulated learning behaviours and strategies to help support development of adaptive lifelong learners.

Perceptions of family medicine in Canada through the eyes of learners.

OBJECTIVE: To examine the perceptions of family medicine (FM) residents about their chosen specialty and how they perceive that patients, other specialists, and the government value FM. DESIGN: Self-report data from the Family Medicine Longitudinal Survey collected from 2014 (time 1 [T1]) to 2016 (time 2 [T2]). SETTING: Canada. PARTICIPANTS: Family medicine residents from 16 out of the 17 FM residency programs. MAIN OUTCOME MEASURES: Responses to statements in the survey were evaluated using a 5-point Likert scale (from strongly disagree to strongly agree). Data were analyzed in 2 ways: cross sectionally (participation in either T1 or T2), and longitudinally (participation in both T1 and T2). RESULTS: For both the cross-sectional cohorts (T1, n = 916; T2, n = 785) and the repeated-measures cohort (n = 420), most residents responded positively to feeling proud of becoming a family physician, with little change from entrance to exit. For both cohorts, a higher proportion of residents at the end of training reported that other medical specialists value the contributions of family physicians (P < .001); however, fewer believed that the government perceived FM as essential to the health care system (P < .001). CONCLUSION: Most participating Canadian FM residents feel proud to become family physicians. This feeling may come from the perceptions of others who are believed to value FM, including other specialists. Measuring attitudinal perceptions offers a window to discover how FM is viewed and can offer a way to measure the effect of strategies implemented to advance the discipline of FM.

Growth mindset in competency-based medical education.

The ongoing adoption of competency-based medical education (CBME) across health professions training draws focus to learner-centred educational design and the importance of fostering a growth mindset in learners, teachers, and educational programs. An emerging body of literature addresses the instructional practices and features of learning environments that foster the skills and strategies necessary for trainees to be partners in their own learning and progression to competence and to develop skills for lifelong learning. Aligned with this emerging area is an interest in Dweck's self theory and the concept of the growth mindset. The growth mindset is an implicit belief held by an individual that intelligence and abilities are changeable, rather than fixed and immutable. In this paper, we present an overview of the growth mindset and how it aligns with the goals of CBME. We describe the challenges associated with shifting away from the fixed mindset of most traditional medical education assumptions and practices and discuss potential solutions and strategies at the individual, relational, and systems levels. Finally, we present future directions for research to better understand the growth mindset in the context of CBME.

Key considerations in planning and designing programmatic assessment in competency-based medical education.

Programmatic assessment as a concept is still novel for many in clinical education, and there may be a disconnect between the academics who publish about programmatic assessment and the front-line clinical educators who must put theory into practice. In this paper, we clearly define programmatic assessment and present high-level guidelines about its implementation in competency-based medical education (CBME) programs. The guidelines are informed by literature and by lessons learned from established programmatic assessment approaches. We articulate five steps to consider when implementing programmatic assessment in CBME contexts: articulate the purpose of the program of assessment, determine what must be assessed, choose tools fit for purpose, consider the stakes of assessments, and define processes for interpreting assessment data. In the process, we seek to offer a helpful guide or template for front-line clinical educators. We dispel some myths about programmatic assessment to help training programs as they look to design-or redesign-programs of assessment. In particular, we highlight the notion that programmatic assessment is not 'one size fits all'; rather, it is a system of assessment that results when shared common principles are considered and applied by individual programs as they plan and design their own bespoke model of programmatic assessment for CBME in their unique context.

On the validity of summative entrustment decisions.

Health care revolves around trust. Patients are often in a position that gives them no other choice than to trust the people taking care of them. Educational programs thus have the responsibility to develop physicians who can be trusted to deliver safe and effective care, ultimately making a final decision to entrust trainees to graduate to unsupervised practice. Such entrustment decisions deserve to be scrutinized for their validity. This end-of-training entrustment decision is arguably the most important one, although earlier entrustment decisions, for smaller units of professional practice, should also be scrutinized for their validity. Validity of entrustment decisions implies a defensible argument that can be analyzed in components that together support the decision. According to Kane, building a validity argument is a process designed to support inferences of scoring, generalization across observations, extrapolation to new instances, and implications of the decision. A lack of validity can be caused by inadequate evidence in terms of, according to Messick, content, response process, internal structure (coherence) and relationship to other variables, and in misinterpreted consequences. These two leading frameworks (Kane and Messick) in educational and psychological testing can be well applied to summative entrustment decision-making. The authors elaborate the types of questions that need to be answered to arrive at defensible, well-argued summative decisions regarding performance to provide a grounding for high-quality safe patient care.

Questioning medical competence: Should the Covid-19 crisis affect the goals of medical education?

The COVID-19 pandemic has disrupted many societal institutions, including health care and education. Although the pandemic's impact was initially assumed to be temporary, there is growing conviction that medical education might change more permanently. The International Competency-based Medical Education (ICBME) collaborators, scholars devoted to improving physician training, deliberated how the pandemic raises questions about medical competence. We formulated 12 broad-reaching issues for discussion, grouped into micro-, meso-, and macro-level questions. At the individual micro level, we ask questions about adaptability, coping with uncertainty, and the value and limitations of clinical courage. At the institutional meso level, we question whether curricula could include more than core entrustable professional activities (EPAs) and focus on individualized, dynamic, and adaptable portfolios of EPAs that, at any moment, reflect current competence and preparedness for disasters. At the regulatory and societal macro level, should conditions for licensing be reconsidered? Should rules of liability be adapted to match the need for rapid redeployment? We do not propose a blueprint for the future of medical training but rather aim to provoke discussions needed to build a workforce that is competent to cope with future health care crises.

Clarifying essential terminology in entrustment.

With the rapid uptake of entrustable professional activties and entrustment decision-making as an approach in undergraduate and graduate education in medicine and other health professions, there is a risk of confusion in the use of new terminologies. The authors seek to clarify the use of many words related to the concept of entrustment, based on existing literature, with the aim to establish logical consistency in their use. The list of proposed definitions includes independence, autonomy, supervision, unsupervised practice, oversight, general and task-specific trustworthiness, trust, entrust(ment), entrustable professional activity, entrustment decision, entrustability, entrustment-supervision scale, retrospective and prospective entrustment-supervision scales, and entrustment-based discussion. The authors conclude that a shared understanding of the language around entrustment is critical to strengthen bridges among stages of training and practice, such as undergraduate medical education, graduate medical education, and continuing professional development. Shared language and understanding provide the foundation for consistency in interpretation and implementation across the educational continuum.