'We're like Spider-Man; with great power comes great responsibility': Coaches' experiences supporting struggling medical students.

BACKGROUND: Medical students can experience a range of academic and non-academic struggles. Coaching is a valuable strategy to support learners, but coaches describe working with struggling learners as taxing. Transformative learning theory (TLT) provides insights into how educators grow from challenging experiences to build resilience. This study explores how coaches evolve as educators through supporting struggling students. METHODS: This qualitative study grounded in an interpretivist paradigm used interviews of longitudinal medical student coaches at two academic institutions. Interviews, using TLT as a sensitizing concept, explored coaches' experience coaching struggling learners. We performed thematic analysis. RESULTS: We interviewed 15 coaches. Coaches described supporting students through multi-faceted struggles which often surprised the coach. Three themes characterized coaches' experiences: personal responsibility, emotional response, and personal learning. Coaches shouldered high personal responsibility for learners' success. For some, this burden felt emotional, raised parental instincts and questions about maintaining boundaries with learners. Coaches evolved their coaching approach, challenged biases, and built skills. Coaches learned to better appreciate the learner point of view and employ resources to support students. DISCUSSION: Through navigating learner struggles, educators can gain self-efficacy, learn to understand learners' perspectives, and evolve their coaching approach to lessen their personal emotional burden through time.

Teaching the tutors: use of an OSTE to train medical students to be peer tutors.

First-year medical students are often challenged by the rapid pace and large volume of content that must be learned. Peer teaching has emerged as a supportive educational strategy. However, the most effective strategies for training peer tutors (PTs) for their role are not known. This paper examines the use of an Objective Structured Teaching Exercise (OSTE) to augment PT training sessions. Applying deliberate practice as a conceptual framework, an OSTE was used to provide tutors with an opportunity to practice their skills and receive feedback about their performance when meeting with a student presenting with a challenge. The newly trained PTs were required to assess a standardized student, determine challenge(s) being experienced, and present options to address the challenge(s). Standardized students evaluated the tutors' performance and a pre- and post-OSTE questionnaire was used to determine whether the OSTE was effective in increasing the confidence level of PTs to effectively assess and support students seeking help. Participants reported an increase in confidence in their ability to assess areas requiring improvement, understand the active learning strategies, and suggest appropriate active learning study strategies. Evaluations completed by standardized students documented that newly trained PTs accurately diagnosed the challenge presented in the OSTE and in most cases PTs asked all relevant questions to assess. Increased self-efficacy promotes PT's capacity to perform their work and feedback during an OSTE can further advance required skills. Aggregate OSTE results can also inform efforts to enhance the PT training program.NEW & NOTEWORTHY This novel application of the Objective Structured Teaching Exercise (OSTE) was done to enhance tutors' skills as valued members of our integrated academic support program. The OSTE provided feedback to the tutors and enabled us to identify a need for enhanced tutor training in active learning strategies. The OSTE can be adapted for use in other health science educational programs to enhance their training programs and to assess tutor's skills in preparation for their role.

A novel virtual course to teach medical students high-value decision-making.

BACKGROUND: Although a clinician's ability to employ high-value decision-making is influenced by training, many undergraduate medical education programmes lack a formal curriculum in high-value, cost-conscious care. We present a curriculum developed through a cross-institutional collaboration that was used to teach students at two institutions about this topic and can serve as a framework for other institutions to develop similar curricula. APPROACH: The faculty from the University of Virginia and the Johns Hopkins University School of Medicine created a 2-week-long online course to teach medical students the fundamentals of high-value care. The course consisted of learning modules, clinical cases, textbook studies, journal clubs and a competitive 'Shark Tank' final project where students proposed a realistic intervention to promote high-value clinical care. EVALUATION: Over two-thirds of students rated the course's quality as excellent or very good. Most found the online modules (92%), assigned textbook readings (89%) and 'Shark Tank' competition (83%) useful. To evaluate the student's ability to apply the concepts learned during the course in clinical contexts, we developed a scoring rubric based on the New World Kirkpatrick Model to evaluate students' proposals. Groups chosen as finalists (as determined by faculty judges) were more likely to be fourth-year students (56%), achieved higher overall scores (p = 0.03), better incorporated cost impact at several levels (patient, hospital and national) (p = 0.001) and discussed both positive and negative impacts on patient safety (p = 0.04). IMPLICATIONS: This course provides a framework for medical schools to use in their teaching of high-value care. Cross-institutional collaboration and online content overcame local barriers such as contextual factors and lack of faculty expertise, allowed for greater flexibility, and enabled focused curricular time to be spent on a capstone project competition. Prior clinical experience amongst medical students may be an enabling factor in promoting application of learning related to high-value care.

Becoming by doing: How women in academic health sciences build a leadership identity through project experiences.

PURPOSE: Leadership development programs often use institutional projects to activate learning. We explored how project work shaped leadership identity formation in senior women leaders from one academic health science center who enrolled in The Hedwig von Ameringen Executive Leadership in Academic Medicine (ELAM®) program. MATERIALS AND METHODS: We interviewed ELAM Fellows and conducted a qualitative analysis of transcripts. Our primary analysis focused on the influences of projects on Fellows and institutions. Leadership identity formation emerged as a distinct pattern, so this narrative content was separated for secondary analysis. All authors approved the final assignment of themes and codes. RESULTS: Participants described a multi-dimensional process for developing a leadership identity. Themes encompassed participants' View of Self and One's Image as a Leader, Interpersonal Relationships, and Commitment to a Value-based Goal. These internal factors grounded external influences, such as interactions with colleagues and institutional leaders, and the world beyond the institution. CONCLUSIONS: We examined the process of leadership identity formation from the perspective of women leaders in academic health sciences who completed an institutional project during a leadership development program. Findings illustrate how internal and external forces, experienced in the context of project work, combine to influence leadership identity formation in women.

Fidelity to Best Practices in EPA Implementation: Outcomes Supporting Use of the Core Components Framework From the University of Virginia Entrustable Professional Activity Program.

PROBLEM: The rapid expansion of entrustable professional activity (EPA) assessment programs has led to calls to ensure fidelity in implementation and integrity in meeting the goals of competency-based medical education. Initiated in July 2017, in advance of the articulated core components of EPA implementation, this article describes the structure and outcomes of the University of Virginia (UVA) EPA Program and provides support for the identified essential components. APPROACH: The UVA EPA Program includes workplace assessments by residents/fellows, attending faculty, and master assessors (MAs), experienced clinicians who assess students across disciplines and clinical settings. All assessors participate in formal professional development and provide verbal and written comments to support their supervision ratings. The Entrustment Committee, composed of 12 MAs, uses a shared mental model and aggregates all assessor data to make a high-stakes summative entrustment decision about students' readiness to assume the role of an acting intern. OUTCOMES: Since 2017, over 2,000 assessors have completed 56,969 EPA assessments for 1,479 students. Ninety-four percent of assessments have been done during the clerkship phase. Residents/fellows have completed a mean of 18 assessments, attending faculty a mean of 27, and MAs a mean of 882. Seventy-four percent of observed encounters involved patients with acute concerns with or without a co-morbid condition. Fifty percent of assessments occurred in inpatient and 32% in ambulatory settings. Eighty-seven percent of assessments contained narrative comments with more than 100 characters. NEXT STEPS: Planned next steps will include earlier identification of students who require individualized learning to promote the development of skills related to EPAs, expansion of the remediation program to enable more students to engage in a clinical performance mastery elective, and creation of targeted professional development for assessors to reinforce the tenets of the EPA program.

How Do Clerkship Students Use EPA Data? Illuminating Students' Perspectives as Partners in Programs of Assessment.

INTRODUCTION: The implementation of programs of assessment based on Entrustable Professional Activities (EPAs) offers an opportunity for students to obtain unique data to guide their ongoing learning and development. Although authors have explored factors that contribute to trust-based decisions, learners' use of assessors' decisions about the level of supervision they need has not been fully investigated. METHODS: In this study, we conducted semi-structured interviews of clerkship students who participated in the first year of our EPA program to determine how they interpret and use supervision ratings provided in EPA assessments. Content analysis was performed using concept-driven and open coding. RESULTS: Nine interviews were completed. Twenty-two codes derived from previous work describing factors involved in trust decisions and 12 novel codes were applied to the interview text. Analyses revealed that students focus on written and verbal feedback from assessors more so than on supervision ratings. Axial coding revealed a temporal organization that categorized how students considered the data from EPA assessments. While factors before, during, and after an assessment affected students' use of information, the relationship between the student and the assessor had impact throughout. CONCLUSIONS: Although students reported varying use of the supervision ratings, their perspectives about how assessors and students interact and/or partner before, during, and after assessments provide insights into the importance of an educational alliance in making a program of assessment meaningful and acceptable to learners.

The use of EPA assessments in decision-making: Do supervision ratings correlate with other measures of clinical performance?

BACKGROUND: Entrustable professional activities (EPAs) have been introduced as a framework for teaching and assessment in competency-based educational programs. With growing use, has come a call to examine the validity of EPA assessments. We sought to explore the correlation of EPA assessments with other clinical performance measures to support use of supervision ratings in decisions about medical students' curricular progression. METHODS: Spearman rank coefficients were used to determine correlation of supervision ratings from EPA assessments with scores on clerkship evaluations and performance on an end-of-clerkship-year Objective Structured Clinical Examination (CPX). RESULTS: Both overall clinical evaluation items score (rho 0.40; n = 166) and CPX patient encounter domain score (rho 0.31; n = 149) showed significant correlation with students' overall mean EPA supervision rating during the clerkship year. There was significant correlation between mean supervision rating for EPA assessments of history, exam, note, and oral presentation skills with scores for these skills on clerkship evaluations; less so on the CPX. CONCLUSIONS: Correlation of EPA supervision ratings with commonly used clinical performance measures offers support for their use in undergraduate medical education. Data supporting the validity of EPA assessments promotes stakeholders' acceptance of their use in summative decisions about students' readiness for increased patient care responsibility.

Leadership Development Projects: Bidirectional Impact on Faculty and Institutions.

INTRODUCTION: The Hedwig von Ameringen Executive Leadership in Academic Medicine program (ELAM) is a national professional development program for women that includes institutional action projects (IAPs). Although benefits of ELAM participation are well documented, the value of the IAPs has not been specifically evaluated. We explored the experience of ELAM Fellows and leaders from one institution to elucidate how institutional factors influence project implementation and outcomes. METHODS: Fellows and deans participated in semistructured interviews. We analyzed the transcripts qualitatively to develop themes and describe factors that influenced IAP implementation and outcomes. We used the New World Kirkpatrick Model, an updated version of the widely used Kirkpatrick model of educational program evaluation, as a framework to elucidate how participants applied their leadership learning through project work, and to analyze early results of projects that indicated institutional impact. RESULTS: Project work had bidirectional impact on the fellows in the program and on the institution itself. Project enablers included: focusing projects on institutional priorities, obtaining sustainable support, and navigating institutional complexity. Leading indicators of institutional outcomes included contributions to institutional leadership and culture, and mutual enhancement of the reputation of the fellow and of the institution. DISCUSSION: By examining enablers and barriers for institutionally based projects conducted in a national leadership development program, we identified the drivers that facilitated application of leadership learning. Leading indicators of project outcomes reflected bidirectional impact on fellows and the institution, demonstrating outcomes at the highest levels of the New World Kirkpatrick Model.

You can have both: Coaching to promote clinical competency and professional identity formation.

Coaching is a critical tool to guide student development of clinical competency and formation of professional identity in medicine, two inextricably linked concepts. Because progress toward clinical competence is linked to thinking, acting and feeling like a physician, a coach's knowledge about a learner's development of clinical skills is essential to promoting the learner's professional identity formation. A longitudinal coaching program provides a foundation for the formation of coach-learner relationships built on trust. Trusting relationships can moderate the risk and vulnerability inherent in a hierarchical medical education system and allow coaching conversations to focus on the promotion of self-regulated learning and fostering skills for life-long learning. Herein, we describe a comprehensive, longitudinal clinical coaching program for medical students designed to support learners' professional identify formation and effectively promote their emerging competence.

Implementing Systematic Faculty Development to Support an EPA-Based Program of Assessment: Strategies, Outcomes, and Lessons Learned.

Problem: Development of a novel, competency-based program of assessment requires creation of a plan to measure the processes that enable successful implementation. The principles of implementation science outline the importance of considering key drivers that support and sustain transformative change within an educational program. The introduction of Entrustable Professional Activities (EPAs) as a framework for assessment has underscored the need to create a structured plan to prepare assessors to engage in a new paradigm of assessment. Although approaches to rater training for workplace-based assessments have been described, specific strategies to prepare assessors to apply standards related to the level of supervision a student needs have not been documented. Intervention: We describe our systematic approach to prepare assessors, faculty and postgraduate trainees, to complete EPA assessments for medical students during the clerkship phase of our curriculum. This institution-wide program is designed to build assessors' skills in direct observation of learners during authentic patient encounters. Assessors apply new knowledge and practice skills in using established performance expectations to determine the level of supervision a learner needs to perform clinical tasks. Assessors also learn to provide feedback and narrative comments to coach students and promote their ongoing clinical development. Data visualizations for assessors facilitate reinforcement of the tenets learned during training. Collaborative learning and peer feedback during faculty development sessions promote the formation of a community of practice among assessors. Context: Faculty development for assessors was implemented in advance of implementation of the EPA program. Assessors in the program include residents/fellows who work closely with students, faculty with discipline-specific expertise and a group of experienced clinicians who were selected to serve as experts in competency-based EPA assessments, the Master Assessors. Training focused on creating a shared understanding about the application of criteria used to evaluate student performance. EPA assessments based on the AAMC's Core Entrustable Professional Activities for Entering Residency, were completed in nine core clerkships. EPA assessments included a supervision rating based on a modified scale for use in undergraduate medical education. Impact: Data from EPA assessments completed during the first year of the program were analyzed to evaluate the effectiveness of the faculty development activities implemented to prepare assessors to consistently apply standards for assessment. A systematic approach to training and attention to critical drivers that enabled institution-wide implementation, led to consistency in the supervision rating for students' first EPA assessment completed by any type of assessor, ratings by assessors done within a specific clinical context, and ratings assigned by a group of specific assessors across clinical settings. Lessons learned: A systematic approach to faculty development with a willingness to be flexible and reach potential participants using existing infrastructure, can facilitate assessors' engagement in a new culture of assessment. Interaction among participants during training sessions not only promotes learning but also contributes to community building. A leadership group responsible to oversee faculty development can ensure that the needs of stakeholders are addressed and that a change in assessment culture is sustained.

International Medical Graduates in the Pediatric Workforce in the United States.

BACKGROUND AND OBJECTIVES: To describe the supply, distribution, and characteristics of international medical graduates (IMGs) in pediatrics who provide patient care in the United States. METHODS: Cross-sectional study, combining data from the 2019 Physician Masterfile of the American Medical Association and the Educational Commission for Foreign Medical Graduates database. RESULTS: In total, 92 806 pediatric physicians were identified, comprising 9.4% of the entire US physician workforce. Over half are general pediatricians. IMGs account for 23.2% of all general pediatricians and pediatric subspecialists. Of all IMGs in pediatrics, 22.1% or 4775 are US citizens who obtained their medical degree outside the United States or Canada, and 15.4% (3246) attended medical school in the Caribbean. Fifteen non-US medical schools account for 29.9% of IMGs currently in active practice in pediatrics in the United States. IMGs are less likely to work in group practice or hospital-based practice and are more likely to be employed in solo practice (compared with US medical school graduates). CONCLUSIONS: With this study, we provide an overview of the pediatric workforce, quantifying the contribution of IMGs. Many IMGs are US citizens who attend medical school abroad and return to the United States for postgraduate training. Several factors, including the number of residency training positions, could affect future numbers of IMGs entering the United States. Longitudinal studies are needed to better understand the implications that workforce composition and distribution may have for the care of pediatric patients.

Scholarly Evaluation of Curricula and Educational Programs: Using a Systematic Approach to Produce Publishable Scholarship.

The goal of designing innovative curricula and programs in medical education is to create interventions that will change the attitudes, knowledge, skills, and behaviors of learners in order to prepare them to contribute to the health of patients and communities they serve. Systematic evaluation allows curriculum/program developers to optimize their curriculum/program and ensure that the goals of the curriculum/program are met. While curriculum/program evaluation is distinct from educational research, when approached in a rigorous manner, curriculum/program evaluation can be published as educational scholarship. The goal of our paper is to equip readers with tools to apply a scholarly approach to curriculum/program evaluation to produce findings that will advance the field. We start by defining curricula, educational programs, learner assessment, and evaluation. We then briefly describe examples of best practice models for curriculum/program development that incorporate the critical step of planning for evaluation. Building on published work, we distinguish the use of best practice models and conceptual frameworks to inform curriculum/program development and evaluation. More specifically, we outline steps to plan an evaluation that demonstrates WHAT was effective, describes HOW the curriculum/program contributed to the outcomes achieved, and points to WHY the curriculum/program led to the outcomes observed. We conclude with key considerations for publishing findings of an evaluation, including what to include in each section of a manuscript.

Simulation Training Incorporating a Pulsatile Carotid Endarterectomy Model Results in Increased Procedure-Specific Knowledge, Confidence, and Comfort in Post-graduate Trainees.

OBJECTIVES: Surgical simulation has been used to facilitate the acquisition of vascular surgery skills. However, high cost and limited availability may restrict the use of this educational resource. We report how instruction using a low-cost, pulsatile, carotid endarterectomy (CEA) benchtop surgical simulation model can be used to enhance learners' procedure-specific knowledge, comfort, and confidence in performing the steps of a CEA procedure DESIGN: A single instructor engaged each participant in a one-on-one instructional session during which the instructor demonstrated, and then the participants performed, the steps of a CEA. Participants completed a pre- and postintervention assessment of knowledge and attitudes about preforming a CEA and use of simulation as a learning tool. Postintervention, participants rated the impact of the simulation model on their learning. A Related T-test and Wilcoxin signed Rank Test were used to compare pre- and postintervention results. SETTINGS: University of Virginia Health System, Charlottesville, Virginia. PARTICIPANTS: Seventeen postgraduate trainees. RESULTS: A significant difference was observed in pre- and postknowledge scores (48% vs 91% correct, p < 0.01). Trainee confidence (1.65 vs 2.88, p < 0.01) and comfort (1.59 vs 2.82, p < 0.01) with doing the procedure also increased significantly. Sixteen (94%) responded that use of the simulator was extremely or very important as a tool for learning. All 17 trainees (100%) reported that the simulation experience was either essential or very useful in helping them learn how to perform a CEA. Sixty-five percent responded that they were extremely likely to apply the skills learned during the intervention the next time they performed a CEA. CONCLUSIONS: A low-cost, pulsatile CEA simulation model used as an educational tool increased procedure-specific knowledge, comfort, and confidence among trainees. Learner's increased confidence and affirmation that they are likely to apply the learned skills in a clinical setting support the use of this educational approach to impact trainee behaviors.

Case-Based Teaching: Does the Addition of High-Fidelity Simulation Make a Difference in Medical Students' Clinical Reasoning Skills?

CONTEXT: Situativity theory posits that learning and the development of clinical reasoning skills are grounded in context. In case-based teaching, this context comes from recreating the clinical environment, through emulation, as with manikins, or description. In this study, we sought to understand the difference in student clinical reasoning abilities after facilitated patient case scenarios with or without a manikin. METHODS: Fourth-year medical students in an internship readiness course were randomized into patient case scenarios without manikin (control group) and with manikin (intervention group) for a chest pain session. The control and intervention groups had identical student-led case progression and faculty debriefing objectives. Clinical reasoning skills were assessed after the session using a 64-question script concordance test (SCT). The test was developed and piloted prior to administration. Hospitalist and emergency medicine faculty responses on the test items served as the expert standard for scoring. RESULTS: Ninety-six students were randomized to case-based sessions with (n = 48) or without (n = 48) manikin. Ninety students completed the SCT (with manikin n = 45, without manikin n = 45). A statistically significant mean difference on test performance between the two groups was found (t = 3.059, df = 88, p = .003), with the manikin group achieving higher SCT scores. CONCLUSION: Use of a manikin in simulated patient case discussion significantly improves students' clinical reasoning skills, as measured by SCT. These results suggest that using a manikin to simulate a patient scenario situates learning, thereby enhancing skill development.

Moving Toward Summative Competency Assessment to Individualize the Postclerkship Phase.

In the move toward competency-based medical education, leaders have called for standardization of learning outcomes and individualization of the learning process. Significant progress has been made in establishing defined expectations for the knowledge, skills, attitudes, and behaviors required for successful transition to residency training, but individualization of educational processes to assist learners in reaching these competencies has been predominantly conceptual to date. The traditional time-based structure of medical education has posed a challenge to individualization within the curriculum and has led to more attention on innovations that facilitate transition from medical school to residency. However, a shift of focus to the clerkship-to-postclerkship transition point in the undergraduate curriculum provides an opportunity to determine how longitudinal competency-based assessments can be used to facilitate intentional and individualized structuring of the long-debated fourth year.This Perspective demonstrates how 2 institutions-the University of Virginia School of Medicine and the University of Michigan Medical School-are using competency assessments and applying standardized outcomes in decisions about individualization of the postclerkship learning process. One institution assesses Core Entrustable Professional Activities for Entering Residency, whereas the other has incorporated Accreditation Council for Graduate Medical Education core competencies and student career interests to determine degrees of flexibility in the postclerkship phase. Individualization in addition to continued assessment of performance presents an opportunity for intentional use of curriculum time to develop each student to be competently prepared for the transition to residency.

How a Year of Clinical Experience Affects Students' Perspectives About the Supervision They Need to Perform Entrustable Professional Activities.

Assessment of entrustable professional activities (EPAs) requires a supervisor to determine a learner's need for supervision to perform the tasks. This study examines medical students' perspectives, after completing their core clerkships, about the supervision they need to perform tasks defined by the core EPAs for entering residency. Compared with assertions at the start of the clerkship year, on retrospective self-assessment, students reported needing higher levels of supervision predominantly for activities commonly done during clinical rotations. Students remain confident in their ability to perform tasks with autonomy in the post-clerkship phase, however, raising concern about their abilities to discern limitations and appropriately request supervision.