Protocol for a scoping review study on learning plan use in undergraduate medical education.

BACKGROUND: The current paradigm of competency-based medical education and learner-centredness requires learners to take an active role in their training. However, deliberate and planned continual assessment and performance improvement is hindered by the fragmented nature of many medical training programs. Attempts to bridge this continuity gap between supervision and feedback through learner handover have been controversial. Learning plans are an alternate educational tool that helps trainees identify their learning needs and facilitate longitudinal assessment by providing supervisors with a roadmap of their goals. Informed by self-regulated learning theory, learning plans may be the answer to track trainees' progress along their learning trajectory. The purpose of this study is to summarise the literature regarding learning plan use specifically in undergraduate medical education and explore the student's role in all stages of learning plan development and implementation. METHODS: Following Arksey and O'Malley's framework, a scoping review will be conducted to explore the use of learning plans in undergraduate medical education. Literature searches will be conducted using multiple databases by a librarian with expertise in scoping reviews. Through an iterative process, inclusion and exclusion criteria will be developed and a data extraction form refined. Data will be analysed using quantitative and qualitative content analyses. DISCUSSION: By summarising the literature on learning plan use in undergraduate medical education, this study aims to better understand how to support self-regulated learning in undergraduate medical education. The results from this project will inform future scholarly work in competency-based medical education at the undergraduate level and have implications for improving feedback and supporting learners at all levels of competence. SCOPING REVIEW REGISTRATION: Open Science Framework osf.io/wvzbx.

Development of entrustable professional activities for family medicine in South Africa.

South Africa is undergoing a significant shift towards implementing enhanced workplace-based assessment methodologies across various specialist training programmes, including family medicine. This paradigm involves the evaluation of Entrustable Professional Activities (EPAs) through comprehensive portfolios of evidence, which a local and national clinical competency committee then assesses. The initial phase of this transformative journey entails the meticulous development of EPAs rooted in discrete units of work. Each EPA delineates the registrar's level of entrustment for autonomous practice, along with the specific supervision requirements. This concise report details the collaborative effort within the discipline of family medicine in South Africa, culminating in the consensus formation of 22 meticulously crafted EPAs for postgraduate family medicine training. The article intricately outlines the systematic structuring and rationale behind the EPAs, elucidating the iterative process employed in their development. Notably, this marks a groundbreaking milestone as the first comprehensive documentation of EPAs nationally for family medicine training in Africa.

A Scoping Review of the Literature on Entrustable Professional Activities in Surgery Residency Programs.

OBJECTIVE: Entrustable professional activities (EPAs) are a crucial component of contemporary postgraduate medical education with many surgery residency programs having implemented EPAs as a competency assessment framework to assess and provide feedback on the performance of their residents. Despite broad implementation of EPAs, there is a paucity of evidence regarding the impact of EPAs on the learners and learning environments. A first step in improving understanding of the use and impact of EPAs is by mapping the rising number of EPA-related publications from the field of surgery. The primary objective of this scoping review is to examine the nature, extent, and range of articles on the development, implementation, and assessment of EPAs. The second objective is to identify the experiences and factors that influence EPA implementation and use in practice in surgical specialties. DESIGN: Scoping review. Four electronic databases (Medline, Embase, Education Source, and ERIC) were searched on January 20, 2022, and then again on July 19, 2023. A quasi-statistical content analysis was employed to quantify and draw meaning from the information related to the development, implementation, assessment, validity, reliability, and experiences with EPAs in the workplace. PARTICIPANTS: A total of 42 empirical and nonempirical articles were included. RESULTS: Four thematic categories describe the topic areas in included articles related to: 1) the development and refinement of EPAs, including the multiple steps taken to develop and refine unique EPAs for surgery residency programs; 2) the methods for implementing EPAs; 3) outcomes of EPA use in practice; 4) barriers, facilitators, and areas for improvement for the implementation and use of EPAs in surgical education. CONCLUSIONS: This scoping review highlights the key trends and gaps from the rapidly increasing number of publications on EPAs in surgery residency, from development to their use in the workplace. Existing EPA studies lack a theoretical and/or conceptual basis; future development and implementation studies should adopt implementation science frameworks to better structure and operationalize EPAs within surgery residency programs.

Implementation and Impact of an Adapted Digital Perioperative Competency-Building Tool to Enhance Teaching, Learning And Feedback in Postgraduate Competency-Based Medical Education.

OBJECTIVES: The purpose of this educational intervention was to introduce, iteratively adapt, and implement a digital formative assessment tool in a surgical speciality. The study also evaluated the intervention's impact on perioperative teaching, learning, feedback, and surgical competency. DESIGN: A participatory action research model with a mixed methods approach. SETTING: This study was performed over 10 months in an institutional hospital in South Africa with a general surgery department. PARTICIPANTS: Twelve supervising surgical trainers/faculty and 12 surgical trainees/residents consented to participate in the intervention. RESULTS: The first 4 months of the intervention focused on relationship building, a multi-stakeholder contextual needs assessment and training sessions to support a shared mindset and shift in the teaching and learning culture. The final adapted perioperative competency-building tool comprised a 23-item assessment with four open-text answers (Table 1). Over the following 6-month period, 48 workplace-based competency-building perioperative evaluations were completed. Most trainees took less than 5 minutes to self-assess (67%) before most trainers (67%) took less than 5 minutes to give oral feedback to the trainee after the perioperative supervised learning encounter. On average, the digital tool took 6 minutes to complete during the bidirectional perioperative teaching and learning encounter with no negative impact on the operational flow. All trainers and trainees reported the training and implementation of the digital tool to be beneficial to teaching, learning, feedback, and the development of surgical competency. Analysis of the completed tools revealed several trainees showing evidence of progression in surgical competency for index procedures within the speciality. The focus groups and interviews also showed a change in the teaching and learning culture: more positively framed, frequent, structured, and specific feedback, improved accountability, and trainee-trainer perioperative readiness for teaching. Highlighted changes included the usefulness of trainee self-assessment before perioperative trainer feedback and the tool's value in improving competency to Kirkpatrick Level 4. CONCLUSION: Implementing an adapted digital Workplace-Based Assessment (WBA) tool using a participatory action research model has proven successful in enhancing the effectiveness of supervised perioperative teaching and learning encounters. This approach has improved teaching and feedback practices, facilitated the development of surgical competency, and ultimately impacted the overall culture to Kirkpatrick level 4. Importantly, it has positively influenced the trainee-trainer relationship dynamic. Based on these positive outcomes, we recommend using this effective method and our relationship-centred framework for implementing formative competency-building tools in future studies. By doing so, larger-scale and successful implementation of Competency-Based Medical Education (CBME) could be achieved in various contexts. This approach can potentially enhance teaching and learning encounters, promote competency development, and improve the overall educational experience for surgical trainees and trainers.

Domains Influencing Faculty Decisions on the Level of Supervision Required for Anesthesiology EPAs with Analysis of Feedback Comments.

OBJECTIVE: The purpose of this qualitative study was to examine responses related to entrustment and feedback comments from an assessment tool. DESIGN: Qualitative analyses using semi-structured interviews and analysis of narrative comments. SETTING: Main hospital OR suite at a large academic medical center. PARTICIPANTS: faculty, and residents who work in the OR suite. RESULTS: Seven of the 14 theoretical domains from the Theoretical Domains Framework were identified as influencing faculty decision on entrustment: knowledge, skills, intention, memory/attention/decision processes, environmental context, and resources, beliefs of capabilities, and reinforcement. The majority (651/1116 (58.4%)) of faculty comments were critical/modest praise and relevant, consistent across all 6 EPAs. The written in feedback comments for all 1,116 Web App EPA assessments yielded a total of 1,599 sub-competency specific responses. These responses were mapped to core competencies, and at least once to 13 of the 23 ACGME subcompetencies. CONCLUSIONS: Domains identified as influencing faculty decision on entrustment were knowledge, skills, intention, memory/attention/decision processes, environmental context, and resources, beliefs of capabilities, and reinforcement. Most narrative feedback comments were critical/modest praise and relevant, consistent across each of the EPAs.

[Satisfaction with continuing education at a level 1 trauma center-Results of a survey and development of a competency-based continuing education concept]. / Zufriedenheit mit der Weiterbildung an einem Level-I-Traumazentrum ­ Ergebnisse einer Umfrage und Entwicklung eines kompetenzbasierten Weiterbildungskonzeptes.

BACKGROUND: Structured competency-based training is one of the most frequently articulated wishes of residents. METHODS: A survey of 19 residents was conducted regarding their satisfaction with the resident education at a level 1 trauma center. In this article the development of a revised competency-based education concept was carried out. RESULTS: The survey reflected uncertainty as to whether the current structures could meet the requirements of the residency regulations. The improved competency-based education concept consists of clinical mentoring, competency-based catalogs of learning objectives, regular theoretical and practical workshops as well as regular and structured staff evaluations. CONCLUSION: The education concept presented reflects the attempt to establish a contemporary surgical training program which will be evaluated as it progresses.

Enabling Factors for the Implementation of Competency-Based Curricula in Colleges and Schools of Pharmacy.

OBJECTIVES: To review the implementation drivers of competency-based pharmacy education (CBPE) and provide recommendations for enablers. FINDINGS: Competency-based education is an emerging model in the health professions, focusing on time-variable competency development and achievement compared with a time-bound, course-based, traditional model. CBPE is an outcomes-based organized framework of competencies enabling pharmacists to meet health care and societal needs. However, challenges need to be recognized and overcome for the successful implementation of CBPE. Competency drivers include defining the competencies and roles of stakeholders, developing transparent learning trajectories and aligned assessments, and establishing lifetime development programs for stakeholders. Organization drivers include developing support systems for stakeholders; facilitating connections between all educational experiences; and having transparent assessment plans, policies, and procedures that align with core CBPE precepts, including the sustainability of time-variability. Leadership drivers include establishing growth mindset and facilitating a culture of connection between workplace and educational environments, program advocacy by institutional leaders, accepting failures as part of the process, shifting the organizational culture away from learner differentiation toward competence, and maintaining sufficient administrative capability to support CBPE. SUMMARY: The successful implementation of CBPE involves enabling the competency, organization, and leadership drivers that will lead to program success. More research is needed in the areas of creation, implementation, and assessment of CBPE to determine success in this model. We have reviewed and provided recommendations to enable the drivers of successful implementation of CBPE.

Competence By Design: a transformational national model of time-variable competency-based postgraduate medical education.

Postgraduate medical education is an essential societal enterprise that prepares highly skilled physicians for the health workforce. In recent years, PGME systems have been criticized worldwide for problems with variable graduate abilities, concerns about patient safety, and issues with teaching and assessment methods. In response, competency based medical education approaches, with an emphasis on graduate outcomes, have been proposed as the direction for 21st century health profession education. However, there are few published models of large-scale implementation of these approaches. We describe the rationale and design for a national, time-variable competency-based multi-specialty system for postgraduate medical education called Competence by Design. Fourteen innovations were bundled to create this new system, using the Van Melle Core Components of competency based medical education as the basis for the transformation. The successful execution of this transformational training system shows competency based medical education can be implemented at scale. The lessons learned in the early implementation of Competence by Design can inform competency based medical education innovation efforts across professions worldwide.

Design and Implementation of a National Program of Assessment Model - Integrating Entrustable Professional Activity Assessments in Canadian Specialist Postgraduate Medical Education.

Traditional approaches to assessment in health professions education systems, which have generally focused on the summative function of assessment through the development and episodic use of individual high-stakes examinations, may no longer be appropriate in an era of competency based medical education. Contemporary assessment programs should not only ensure collection of high-quality performance data to support robust decision-making on learners' achievement and competence development but also facilitate the provision of meaningful feedback to learners to support reflective practice and performance improvement. Programmatic assessment is a specific approach to designing assessment systems through the intentional selection and combination of a variety of assessment methods and activities embedded within an educational framework to simultaneously optimize the decision-making and learning function of assessment. It is a core component of competency based medical education and is aligned with the goals of promoting assessment for learning and coaching learners to achieve predefined levels of competence. In Canada, postgraduate specialist medical education has undergone a transformative change to a competency based model centred around entrustable professional activities (EPAs). In this paper, we describe and reflect on the large scale, national implementation of a program of assessment model designed to guide learning and ensure that robust data is collected to support defensible decisions about EPA achievement and progress through training. Reflecting on the design and implications of this assessment system may help others who want to incorporate a competency based approach in their own country.

Exploring the Quality of Feedback in Entrustable Professional Activity Narratives Across 24 Residency Training Programs.

Background Competency-based medical education (CBME) has been implemented in many residency training programs across Canada. A key component of CBME is documentation of frequent low-stakes workplace-based assessments to track trainee progression over time. Critically, the quality of narrative feedback is imperative for trainees to accumulate a body of evidence of their progress. Suboptimal narrative feedback will challenge accurate decision-making, such as promotion to the next stage of training. Objective To explore the quality of documented feedback provided on workplace-based assessments by examining and scoring narrative comments using a published quality scoring framework. Methods We employed a retrospective cohort secondary analysis of existing data using a sample of 25% of entrustable professional activity (EPA) observations from trainee portfolios from 24 programs in one institution in Canada from July 2019 to June 2020. Statistical analyses explore the variance of scores between programs (Kruskal-Wallis rank sum test) and potential associations between program size, CBME launch year, and medical versus surgical specialties (Spearman's rho). Results Mean quality scores of 5681 narrative comments ranged from 2.0±1.2 to 3.4±1.4 out of 5 across programs. A significant and moderate difference in the quality of feedback across programs was identified (χ2=321.38, P<.001, ε2=0.06). Smaller programs and those with an earlier launch year performed better (P<.001). No significant difference was found in quality score when comparing surgical/procedural and medical programs that transitioned to CBME in this institution (P=.65). Conclusions This study illustrates the complexity of examining the quality of narrative comments provided to trainees through EPA assessments.

Learner Assessment and Program Evaluation: Supporting Precision Education.

ABSTRACT: Precision education (PE) systematically leverages data and advanced analytics to inform educational interventions that, in turn, promote meaningful learner outcomes. PE does this by incorporating analytic results back into the education continuum through continuous feedback cycles. These data-informed sequences of planning, learning, assessing, and adjusting foster competence and adaptive expertise. PE cycles occur at individual (micro), program (meso), or system (macro) levels. This article focuses on program- and system-level PE.Data for PE come from a multitude of sources, including learner assessment and program evaluation. The authors describe the link between these data and the vital role evaluation plays in providing evidence of educational effectiveness. By including prior program evaluation research supporting this claim, the authors illustrate the link between training programs and patient outcomes. They also describe existing national reports providing feedback to programs and institutions, as well as 2 emerging, multiorganization program- and system-level PE efforts. The challenges encountered by those implementing PE and the continuing need to advance this work illuminate the necessity for increased cross-disciplinary collaborations and a national cross-organizational data-sharing effort.Finally, the authors propose practical approaches for funding a national initiative in PE as well as potential models for advancing the field of PE. Lessons learned from successes by others illustrate the promise of these recommendations.

Using Natural Language Processing to Evaluate the Quality of Supervisor Narrative Comments in Competency-Based Medical Education.

PURPOSE: Learner development and promotion rely heavily on narrative assessment comments, but narrative assessment quality is rarely evaluated in medical education. Educators have developed tools such as the Quality of Assessment for Learning (QuAL) tool to evaluate the quality of narrative assessment comments; however, scoring the comments generated in medical education assessment programs is time intensive. The authors developed a natural language processing (NLP) model for applying the QuAL score to narrative supervisor comments. METHOD: Samples of 2,500 Entrustable Professional Activities assessments were randomly extracted and deidentified from the McMaster (1,250 comments) and Saskatchewan (1,250 comments) emergency medicine (EM) residency training programs during the 2019-2020 academic year. Comments were rated using the QuAL score by 25 EM faculty members and 25 EM residents. The results were used to develop and test an NLP model to predict the overall QuAL score and QuAL subscores. RESULTS: All 50 raters completed the rating exercise. Approximately 50% of the comments had perfect agreement on the QuAL score, with the remaining resolved by the study authors. Creating a meaningful suggestion for improvement was the key differentiator between high- and moderate-quality feedback. The overall QuAL model predicted the exact human-rated score or 1 point above or below it in 87% of instances. Overall model performance was excellent, especially regarding the subtasks on suggestions for improvement and the link between resident performance and improvement suggestions, which achieved 85% and 82% balanced accuracies, respectively. CONCLUSIONS: This model could save considerable time for programs that want to rate the quality of supervisor comments, with the potential to automatically score a large volume of comments. This model could be used to provide faculty with real-time feedback or as a tool to quantify and track the quality of assessment comments at faculty, rotation, program, or institution levels.

Promotion in Place: A Model for Competency-Based, Time-Variable Graduate Medical Education.

PROBLEM: Competency-based medical education is increasingly regarded as a preferred framework for physician training, but implementation is limited. U.S. residency programs remain largely time based, with variable assessments and limited opportunities for individualization. Gaps in graduates' readiness for unsupervised care have been noted across specialties. Logistical barriers and regulatory requirements constrain movement toward competency-based, time-variable (CBTV) graduate medical education (GME), despite its theoretical benefits. APPROACH: The authors describe a vision for CBTV-GME and an implementation model that can be applied across specialties. Termed "Promotion in Place" (PIP), the model relies on enhanced assessment, clear criteria for advancement, and flexibility to adjust individuals' responsibilities and time in training based on demonstrated competence. PIP allows a resident's graduation to be advanced or delayed accordingly. Residents deemed competent for early graduation can transition to attending physician status within their training institution and benefit from a period of "sheltered independence" until the standard graduation date. Residents who need extended time to achieve competency have graduation delayed to incorporate additional targeted education. OUTCOMES: A proposal to pilot the PIP model of CBTV-GME received funding through the American Medical Association's "Reimagining Residency" initiative in 2019. Ten of 46 residency programs in a multihospital system expressed interest and pursued initial planning. Seven programs withdrew for reasons including program director transitions, uncertainty about resident reactions, and the COVID-19 pandemic. Three programs petitioned their specialty boards for exemptions from time-based training. One program was granted the needed exemption and launched a PIP pilot, now in year 4, demonstrating the feasibility of implementing this model. Implementation tools and templates are described. NEXT STEPS: Larger-scale implementation with longer-term assessment is needed to evaluate the impact and generalizability of this CBTV-GME model.

Development, implementation, and evaluation of entrustable professional activities (EPAs) for medical radiation technologists in Taiwan: a nationwide experience.

BACKGROUND: Competency-based medical education (CBME) is an outcomes-oriented approach focused on developing competencies that translate into clinical practice. Entrustable professional activities (EPAs) bridge competency assessment and clinical performance by delineating essential day-to-day activities that can be entrusted to trainees. EPAs have been widely adopted internationally, but not yet implemented for medical radiation professionals in Taiwan. MATERIALS AND METHODS: A nationwide consensus process engaged 97 experts in radiation technology education representing diagnostic radiography, radiation therapy, and nuclear medicine. Preliminary EPAs were developed through the focus group discussion and the modified Delphi method. The validity of these EPAs was evaluated using the QUEPA and EQual tools. RESULTS: Through iterative consensus building, six core EPAs with 18 component observable practice activities (OPAs) in total were developed, encompassing routines specific to each radiation technology specialty. QUEPA and EQual questionnaire data verified these EPAs were valid, and of high quality for clinical teaching and evaluation. CONCLUSION: The consensus development of tailored EPAs enables rigorous competency assessment during medical radiation technology education in Taiwan. Further expansion of EPAs and training of clinical staff could potentially enhance care quality by producing competent professionals.

Exploring Competency-Based Medical Education Through the Lens of the UME-GME Transition: A Qualitative Study.

PURPOSE: Competency-based medical education (CBME) represents a shift to a paradigm with shared definitions, explicit outcomes, and assessments of competence. The groundwork has been laid to ensure all learners achieve the desired outcomes along the medical education continuum using the principles of CBME. However, this continuum spans the major transition from undergraduate medical education (UME) to graduate medical education (GME) that is also evolving. This study explores the experiences of medical educators working to use CBME assessments in the context of the UME-GME transition and their perspectives on the existing challenges. METHOD: This study used a constructivist-oriented qualitative methodology. In-depth, semistructured interviews of UME and GME leaders in CBME were performed between February 2019 and January 2020 via Zoom. When possible, each interviewee was interviewed by 2 team members, one with UME and one with GME experience, which allowed follow-up questions to be pursued that reflected the perspectives of both UME and GME educators more fully. A multistep iterative process of thematic analysis was used to analyze the transcripts and identify patterns across interviews. RESULTS: The 9 interviewees represented a broad swath of UME and GME leadership positions, though most had an internal medicine training background. Analysis identified 4 overarching themes: mistrust (a trust chasm exists between UME and GME); misaligned goals (the residency selection process is antithetical to CBME); inadequate communication (communication regarding competence is infrequent, often unidirectional, and lacks a shared language); and inflexible timeframes (current training timeframes do not account for individual learners' competency trajectories). CONCLUSIONS: Despite the mutual desire and commitment to move to CBME across the continuum, mistrust, misaligned goals, inadequate communication, and inflexible timeframes confound such efforts of individual schools and programs. If current efforts to improve the UME-GME transition address the themes identified, educators may be more successful implementing CBME along the continuum.

Do summative entrustment decisions actually lead to entrustment?

BACKGROUND: Entrustable professional activities (EPAs) were introduced across Dutch postgraduate programmes between 2017 and 2019. We aimed to understand the extent to which residents actually were granted increased clinical responsibility upon receiving summative entrustment for an EPA, a critical feature of its use. METHODS: A survey study was conducted among all Dutch residents who started dermatology training in 2018 and 2019 and all Dutch dermatology programme directors (PDs). We chose an EPA designed for early entrustment in residency (identification, treatment and care regarding a simple dermatological problem in the ambulatory setting). The survey contained two hypothetical clinical cases that aligned with this EPA. The questions were aimed to determine whether and when residents should request supervision. Similar questions were posed to PDs. FINDINGS: Twenty four residents (56%) and 19 PDs (79%) completed the survey. The majority of the residents (65%) and PDs (63%) confirmed that competent dermatology residents (level 4) are generally allowed to perform EPA1 unsupervised, particularly when seeing patients from GPs. However, still a substantial proportion of the level 4 residents, working in University Medical Centers (36%) indicated that they had to request supervision in the assessment of these patients. For 2nd opinions, the results were typically the opposite. DISCUSSION AND CONCLUSION: This study demonstrated that, at least in one specialty and one country, the introduction of EPAs and entrustment decision making procedure generally led to the intended autonomy of the resident.

The engagement chasm: Time for competency-based training in employee engagement for healthcare and beyond.

BACKGROUND: Employee disengagement in healthcare and business is currently at unexceptionally high levels worldwide. Disengagement negatively impacts productivity, profitability, efficiency (waste reduction), innovation, quality, customer satisfaction and experience, staff well-being, safety, mortality, staff attendance, and turnover. Despite its serious detrimental impacts, no dedicated competency-based training curriculum exists for engagement as a competency. OBJECTIVE: To develop a competency-based educational curriculum for an Engagement Competency. METHODS: A curricular roadmap comprising the following steps was observed 1. Identifying the desired outcomes needed of trainees. These must help fulfill all the Q12 Gallup survey engagement items. 2. Explicitly defining the required Competencies, Entrustable Professional Activities, and Milestones, 3. Selecting the educational activities, and instructional methods, 4. Selecting the tools to assess progress along the milestones, and finally, 5. Designing an evaluation system to assess the outcomes of the engagement competency program. RESULTS: We developed an Engagement Competency Framework with 7 Entrustable Professional activities "rationally and practically" arranged. These are: Envision E1, Embrace E2, Empower E3, Enlighten E4, Empathize E5, Energize E6, and Evaluate E7 (the 7Es). CONCLUSION: The unfortunate global issue of disengagement in healthcare and in the business arena may be practically tackled by introducing Engagement Competency and training. It should be compulsory for all in the "leadership role". Such training may lead to remarkable performance improvement and a happier, more prosperous, and safer world.

Beyond the Procedure Log: Using Individualized Learning Plans to Set Learner-Specific Milestones for Procedural Skills Acquisition.

ABSTRACT: Procedural training for nonsurgical fields, such as internal medicine, is an important component of medical education. However, recent changes to accreditation guidelines have resulted in less formal guidance on procedural competency, not only leading to opportunities for individualizing training but also creating potential problems for trainees and training programs. In this article, the authors use internal medicine as an exemplar to review current strategies for procedural education in nonsurgical fields, including procedural simulation, dedicated procedural rotations, and advanced subspecialty training, and highlight an emerging need for learner-specific terminal milestones in procedural training. Individualized learning plans (ILPs), collections of trainee-specific objectives for learning, are arguably a useful strategy for organizing procedural training. The role of ILPs as a framework to support setting learner-specific terminal milestones, guide skill acquisition, and allocate procedural learning opportunities based on trainees' anticipated career plans is subsequently explored, and how an ILP-based approach might be implemented within the complex educational milieu of a clinical training program is examined. The limitations and pitfalls of an ILP-based approach, including the need for development of coaching programs, are considered. The authors conclude that, despite the limitations of ILPs, when combined with other current strategies for building trainees' procedural competence, these plans may help trainees maximize the educational benefits of their training period and can encourage effective, safer, and equitable allocation of procedural practice opportunities.

Evaluating a Shared Reflective Practice to Develop Nursing Student Clinical Judgment.

BACKGROUND: Self-reflection is a valuable method that nurse educators can use to develop clinical judgment skills among prelicensure nursing students. Little research exists on improving clinical judgment in second-degree nursing students in the clinical setting. PURPOSE: To determine the implications of increasing clinical judgment skills in prelicensure nursing students using a shared structured reflection teaching innovation within a required baccalaureate clinical course. METHODS: This educational innovation used qualitative descriptive methods to evaluate its effectiveness in the clinical setting among students in the final semester of an accelerated prelicensure nursing program. RESULTS: Students reported increased accountability for their learning, a sense of intentionality with their nursing practice, and a deeper sense of community with their peers. CONCLUSIONS: Shared reflective practices in the clinical setting show promise for increasing clinical judgment and supporting a competency-based curriculum.

Deidentifying Narrative Assessments to Facilitate Data Sharing in Medical Education.

PROBLEM: Narrative assessments are commonly incorporated into competency-based medical education programs. However, efforts to share competency-based medical education assessment data among programs to support the evaluation and improvement of assessment systems have been limited in part because of security concerns. Deidentifying assessment data mitigates these concerns, but deidentifying narrative assessments is time-consuming, resource intensive, and error prone. The authors developed and tested a tool to automate the deidentification of narrative assessments and facilitate their review. APPROACH: The authors met throughout 2021 and 2022 to iteratively design, test, and refine the deidentification algorithm and data review interface. Preliminary testing of the prototype deidentification algorithm was performed using narrative assessments from the University of Saskatchewan emergency medicine program. The algorithm's accuracy was assessed by the authors using the review interface designed for this purpose. Formal testing included 2 rounds of deidentification and review by members of the authorship team. Both the algorithm and data review interface were refined during the testing process. OUTCOMES: Authors from 3 institutions, including 3 emergency medicine programs, an anesthesia program, and a surgical program, participated in formal testing. In the final round of review, 99.4% of the narrative assessments were fully deidentified (names, nicknames, and pronouns removed). The results were comparable for each institution and specialty. The data review interface was improved with feedback obtained after each round of review and found to be intuitive. NEXT STEPS: This innovation has demonstrated viability evidence of an algorithmic approach to the deidentification of assessment narratives while reinforcing that a small number of errors are likely to persist. Future steps include the refinement of both the algorithm to improve its accuracy and the data review interface to support additional data set formats.