Clinician Educator Milestones: Assessing and Improving Educators' Skills.

ABSTRACT: The importance of the clinician educator (CE) role in delivery of competency-based medical education is well recognized. There is, however, no formal mechanism to identify when faculty have the knowledge, skills, and attitudes to be successful CEs. In 2020, the Accreditation Council for Graduate Medical Education, Accreditation Council for Continuing Medical Education, Association of American Medical Colleges, and American Association of Colleges of Osteopathic Medicine convened a workgroup of 18 individuals representing multiple medical specialties and diverse institutions in the United States, including nonphysician educators, a medical student, and a resident, to develop a set of competencies, subcompetencies, and milestones for CEs.A 5-step process was used to create the Clinician Educator Milestones (CEMs). In step 1, the workgroup developed an initial CEM draft. Through brainstorming, 141 potential education-related CE tasks were identified. Descriptive statements for each competency and developmental trajectories for each subcompetency were developed and confirmed by consensus. The workgroup then created a supplemental guide, assessment tools, and additional resources. In step 2, a diverse group of CEs were surveyed in 2021 and provided feedback on the CEMs. In step 3, this feedback was used by the workgroup to refine the CEMs. In step 4, the second draft of the CEMs was submitted for public comment, and the CEMs were finalized. In step 5, final CEMs were released for public use in 2022.The CEMs consist of 1 foundational domain that focuses on commitment to lifelong learning, 4 additional domains of competence for CEs in the learning environment, and 20 subcompetencies. These milestones have many potential uses for CEs, including self-assessment, constructing learning and improvement plans, and designing systematic faculty development efforts. The CEMs will continue to evolve as they are applied in practice and as the role of CEs continues to grow and develop.

Competency-Based Time-Variable Anesthesiology Residency Training: Identification of Problems and Solutions.

BACKGROUND: Global medical education is gradually moving toward more comprehensive implementations of a competency-based education (CBE) model. Elimination of standard time-based training and adoption of time-variable training (competency-based time-variable training [CB-TVT]) is one of the final stages of implementation of CBE. While CB-TVT has been implemented in some programs outside the United States, residency programs in the United States are still exploring this approach to training. The Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Medical Specialties (ABMS) are encouraging member boards and residency review committees to consider innovative ways programs could implement CB-TVT. The goals of this study were to (1) identify potential problems with the implementation of CB-TVT in anesthesiology residency training, (2) rank the importance of the problems and the perceived difficulty of solving them, and (3) develop proposed solutions to the identified problems. METHODS: Study participants were recruited from key stakeholder groups in anesthesiology education, including current or former program directors, department chairs, residents, fellows, American Board of Anesthesiology (ABA) board members, ACGME residency review committee members or ACGME leaders, designated institutional officials, residency program coordinators, clinical operations directors, and leaders of large anesthesiology community practice groups. This study was conducted in 2 phases. In phase 1, survey questionnaires were iteratively distributed to participants to identify problems with the implementation of CB-TVT. Participants were also asked to rank the perceived importance and difficulty of each problem and to identify relevant stakeholder groups that would be responsible for solving each problem. In phase 2, surveys focused on identifying potential solutions for problems identified in phase 1. RESULTS: A total of 36 stakeholders identified 39 potential problems, grouped into 7 major categories, with the implementation of CB-TVT in anesthesiology residency training. Of the 39 problems, 19 (48.7%) were marked as important or very important on a 5-point scale and 12 of 19 (63.2%) of the important problems were marked as difficult or very difficult to solve on a 5-point scale. Stakeholders proposed 165 total solutions to the identified problems. CONCLUSIONS: CB-TVT is a promising educational model for anesthesiology residency, which potentially results in learner flexibility, individualization of curricula, and utilization of competencies to determine learner advancement. Because of the potential problems with the implementation of CB-TVT, it is important for future pilot implementations of CB-TVT to document realized problems, efficacy of solutions, and effects on educational outcomes to justify the burden of implementing CB-TVT.

Development and Pilot Testing of a Programmatic System for Competency Assessment in US Anesthesiology Residency Training.

BACKGROUND: In 2018, a set of entrustable professional activities (EPAs) and procedural skills assessments were developed for anesthesiology training, but they did not assess all the Accreditation Council for Graduate Medical Education (ACGME) milestones. The aims of this study were to (1) remap the 2018 EPA and procedural skills assessments to the revised ACGME Anesthesiology Milestones 2.0, (2) develop new assessments that combined with the original assessments to create a system of assessment that addresses all level 1 to 4 milestones, and (3) provide evidence for the validity of the assessments. METHODS: Using a modified Delphi process, a panel of anesthesiology education experts remapped the original assessments developed in 2018 to the Anesthesiology Milestones 2.0 and developed new assessments to create a system that assessed all level 1 through 4 milestones. Following a 24-month pilot at 7 institutions, the number of EPA and procedural skill assessments and mean scores were computed at the end of the academic year. Milestone achievement and subcompetency data for assessments from a single institution were compared to scores assigned by the institution's clinical competency committee (CCC). RESULTS: New assessment development, 2 months of testing and feedback, and revisions resulted in 5 new EPAs, 11 nontechnical skills assessments (NTSAs), and 6 objective structured clinical examinations (OSCEs). Combined with the original 20 EPAs and procedural skills assessments, the new system of assessment addresses 99% of level 1 to 4 Anesthesiology Milestones 2.0. During the 24-month pilot, aggregate mean EPA and procedural skill scores significantly increased with year in training. System subcompetency scores correlated significantly with 15 of 23 (65.2%) corresponding CCC scores at a single institution, but 8 correlations (36.4%) were <30.0, illustrating poor correlation. CONCLUSIONS: A panel of experts developed a set of EPAs, procedural skill assessment, NTSAs, and OSCEs to form a programmatic system of assessment for anesthesiology residency training in the United States. The method used to develop and pilot test the assessments, the progression of assessment scores with time in training, and the correlation of assessment scores with CCC scoring of milestone achievement provide evidence for the validity of the assessments.

Leveraging existing education innovations to establish a community of practice to promote medical education scholar development.

PROBLEM: While some academic health centers have organizational structures to support educational scholarship, such as Medical Education Research Units (MERU), many education scholars may lack access to such institutional resources to support their research agenda and professional growth. APPROACH: In 2014, as part of an externally funded education grant, three faculty educators established a unique education data management system Research & Evaluation Data for Educational Improvement (REDEI). Lacking an institutional MERU, they established an educational research community of practice (CoP) using REDEI as the research dataset. The senior faculty member's effort to facilitate the group is funded by the Dean's Office. OUTCOMES: The CoP meets every 2 weeks to generate research ideas, discuss analytic approach and strategy, review analyses designed to address or explore a research question, and plans for manuscript development. Our CoP has grown from 3 to 18 members representing faculty educators, administrators, and staff across many departments in the School of Medicine. As of 2021, the REDEI system contains performance data on 1,246 students across all years of undergraduate medical education. To date, we have published 11 peer-reviewed educational research manuscripts. Five learners have served as coauthors (three medical students and two residents), three of whom were first authors. Eleven additional papers are in process. This community of practice supports productivity, provides mentorship, overcomes barriers, and is flexible enough for people to join when they can or when an area of interest is actively under development. NEXT STEPS: We are working on educational interdisciplinary research grant submission and creating collaborations with other institutions. Our focus remains on honing skills in grantsmanship, identification of impactful research questions, application of rigorous methods and instrumentation to address them, and refining process of budget development, timelines, and other planning strategies.

Beyond advising and mentoring: Competencies for coaching in medical education.

BACKGROUND: Coaching supports academic goals, professional development and wellbeing in medical education. Scant literature exists on training and assessing coaches and evaluating coaching programs. To begin filling this gap, we created a set of coach competencies for medical education using a modified Delphi approach. METHODS: An expert team assembled, comprised of seven experts in the field of coaching. A modified Delphi approach was utilized to develop competencies. RESULTS: Fifteen competencies in five domains resulted: coaching process and structure, relational skills, coaching skills, coaching theories and models, and coach development. CONCLUSION: These competencies delineate essential features of a coach in medical education. Next steps include creating faculty development and assessment tools for coaching.

Development and Pilot Testing of Entrustable Professional Activities for US Anesthesiology Residency Training.

BACKGROUND: Modern medical education requires frequent competency assessment. The Accreditation Council for Graduate Medical Education (ACGME) provides a descriptive framework of competencies and milestones but does not provide standardized instruments to assess and track trainee competency over time. Entrustable professional activities (EPAs) represent a workplace-based method to assess the achievement of competency milestones at the point-of-care that can be applied to anesthesiology training in the United States. METHODS: Experts in education and competency assessment were recruited to participate in a 6-step process using a modified Delphi method with iterative rounds to reach consensus on an entrustment scale, a list of EPAs and procedural skills, detailed definitions for each EPA, a mapping of the EPAs to the ACGME milestones, and a target level of entrustment for graduating US anesthesiology residents for each EPA and procedural skill. The defined EPAs and procedural skills were implemented using a website and mobile app. The assessment system was piloted at 7 anesthesiology residency programs. After 2 months, faculty were surveyed on their attitudes on usability and utility of the assessment system. The number of evaluations submitted per month was collected for 1 year. RESULTS: Participants in EPA development included 18 education experts from 11 different programs. The Delphi rounds produced a final list of 20 EPAs, each differentiated as simple or complex, a defined entrustment scale, mapping of the EPAs to milestones, and graduation entrustment targets. A list of 159 procedural skills was similarly developed. Results of the faculty survey demonstrated favorable ratings on all questions regarding app usability as well as the utility of the app and EPA assessments. Over the 2-month pilot period, 1636 EPA and 1427 procedure assessments were submitted. All programs continued to use the app for the remainder of the academic year resulting in 12,641 submitted assessments. CONCLUSIONS: A list of 20 anesthesiology EPAs and 159 procedural skills assessments were developed using a rigorous methodology to reach consensus among education experts. The assessments were pilot tested at 7 US anesthesiology residency programs demonstrating the feasibility of implementation using a mobile app and the ability to collect assessment data. Adoption at the pilot sites was variable; however, the use of the system was not mandatory for faculty or trainees at any site.

Transitioning to a new era: Future directions for staff development during COVID-19.

The COVID-19 pandemic created an urgent need for staff development. However, COVID-19 has created many challenges, including the inability to meet in-person, travel restrictions to conferences, overwhelming clinical demands on already overextended faculty members and the increased need to focus on personal health and safety. Although current challenges were immediately met with solutions borne out of an emergency, questions remain on how to identify and sustain best practices and further evolve staff development beyond the immediate crisis. Reviewing the Medical Adaptations series revealed several lessons. Several authors used cognitive apprenticeship to provide scaffolding upon which learners can build skills, knowledge and attitudes. Additionally, moderators were recommended during live educational sessions in order to manage the chat box and engage the audience. Comprehensive IT support was key. A post-session debrief helped deepen understanding and provided a space for peer support and community building. Building a repository for educational materials was recommended. Although we made significant gains in the ability to offer staff development, we must consider potential and unintended consequences and explore how we can use transformative learning theory to capitalize on what we have gained. Utilizing technology can potentially increase access to online learning; however, when not implemented carefully, it can magnify inequities. While providing IT support can serve to mitigate some inequities borne by socioeconomic and generational differences, additional strategies should be implemented to account for English as a second-language learners; those with disabilities who do not have access to adaptive technology; and other marginalized groups who may already feel vulnerable to presenting arguments in oppositions of authority or the majority. Crafting online education experiences to allow for small group, peer-to-peer and social interactions is vital to continued professional and identity development. Now that the urgency has lessened, taking time to ensure what is being offered follows best practices in developing and disseminating quality online education is paramount for broad acceptance.

Trends in medical students' stress, physical, and emotional health throughout training.

Background: Medical student wellness, including physical health, emotional health, and levels of perceived stress, appears to decline during training, with students reporting high levels of depression, anxiety, and burnout as early as the first year of medical school. The impact of curricular changes on health and stress remains unclear, and a modified curriculum that compresses training of the foundational sciences and its effect on wellness has not been studied. Oregon Health & Science University School of Medicine has recently instituted a unique competency-based model, which provides an important opportunity to assess the effects of curricular change on student wellness.Objective: Assess the effects of curricular change on student wellness.Design: Medical students at a single institution were administered the SF-8, an 8-item health-related quality of life survey, as well as the Perceived Stress Scale, a 10-item scale that measures the degree to which life situations are appraised as stressful, at baseline (matriculation) and at the end of Year 1, 2 and 3. Individual variables were assessed over time, as well as a trend analysis of summary domain scores over the 4 time periods.Results: Physical, emotional, and overall health were highest at baseline and lowest at the end of Year 1, after which they improved but never again reached baseline levels. Physical health declined less than emotional health. Perceived stress levels did not change over time but remained moderately high. There were no differences in health or perceived stress based on demographic variables.Conclusions: In a competency-based curriculum, physical, emotional and overall health significantly worsened during Year 1 but improved thereafter, while perceived stress remained unchanged. Early in training, stress and poor overall health may be related to concerns about self-efficacy and workload. Although advanced students show improved wellness, concerns remained about emotional difficulties, such as anxiety and irritability, and feeling a lack of control.

Critical Appraisal of Anesthesiology Educational Research for 2017.

Background Critical appraisals provide a method for establishing the status of an area of study or evaluating the effectiveness of literature within it. The purpose of this study was to review and appraise studies published in 2017 on medical education in anesthesiology and to provide summaries of the highest-quality medical education research articles in the field. Methods Three Ovid MEDLINE databases, Embase.com, Education Resources Information Center (ERIC), and PsycINFO, were searched followed by a manual review of articles published in the highest impact factor journals in both the fields of anesthesiology and medical education. Abstracts were double-screened and quantitative articles subsequently scored by three randomly assigned raters. Qualitative studies were scored by two raters. Two different rubrics were used for scoring quantitative and qualitative studies, both allowed for scores ranging from 1-25. Results A total of 864 unique citations were identified through the search criteria. Of those, 62 articles met the inclusion criteria, with 59 quantitative and three qualitative. The top 10 papers with the highest scores were reported and summarized. Discussion As the first article to critically review the literature available for education in anesthesiology, we hope that this study will serve as the first manuscript in an annual series that will help individuals involved in anesthesiology education gain an understanding of the highest-quality research in the field. Once this process is repeated, trends can be tracked and serve as a resource to educators and researchers in anesthesiology for years to come.

Does Participation in Written Guided Reflective Practice Exercises Affect Readiness for Self-Directed Learning in a Sample of US Anesthesiology Residents?

BACKGROUND: Reflective practice has been identified as one way to increase participation in self-directed lifelong learning so that physicians maintain a level of current and relevant medical knowledge for their practice. This study sought to determine if reflective practice affected the readiness for self-directed learning in a sample of anesthesiology residents in the United States. METHODS: An experimental design was used to employ quantitative methods to investigate the effects of a self-guided 8-week reflective practice exercise on readiness for self-directed learning as measured by Guglielmino's Self-Directed Learning Readiness Scale/Learning Preference Assessment (SDLRS/LPA). Participants were randomly assigned into an experimental group or control group. RESULTS: Fifty-one anesthesiology residents in 3 residency programs completed this study. No significant difference was found between the posttest SDLRS/LPA scores of the control (median = 227) and experimental group (median = 225; U = 294; z = -.584; P = .559; r = 41.18) as well as the pretest and posttest scores (z = -.65; P = .518; r = -.129) of the experimental group. CONCLUSIONS: We should continue to explore ways to train physicians to engage in practices that promote self-directed lifelong learning.