Learning by doing and creation of the shared discovery curriculum.

BACKGROUND: The Michigan State College of Human Medicine began as an experiment to teach medical students in community-based settings and to create a primary care workforce for the state. Decades later, CHM faced internal and external challenges that spurred creation of a new curriculum - the Share Discovery Curriculum - founded on learning by doing and other learning theories. METHODS: A curricular design group (CDG) developed guiding principles for reform. Based on this, pedagogies and structures were selected to achieve this vision and developed into a curricular structure. Components of the first-year curriculum were piloted with a group of students and faculty members. RESULTS: Six guiding principles were endorsed, grounded in learning theories such as Dewey's Learning by Doing. Based upon these, several key features of the new curriculum emerged: learning communities; one-on-one coaches for students; symptom-based presentations for content; simulation, authentic clinical tasks, flipped classrooms, and modified practice-based learning as primary teaching modalities; early, integrated clinical and scientific learning; milestones as course learning objectives; and a multidimensional, competency-based assessment system. DISCUSSION: The process and outcomes described here are intended as an exemplar for schools undertaking curricular change. Early stakeholder engagement, faculty development, sustainable administrative systems, and managing complexity are core to the success of such endeavors.

An Exemplar Milestone Framework for Scaffolding a Competency-Based Medical School Curriculum.

A strong competency and milestone framework is imperative for medical schools adopting competency-based education and assessment. Milestones can be used to align what is taught and what students are asked to demonstrate from matriculation to graduation. We describe the creation and implementation of our milestone framework as an exemplar. Supplementary Information: The online version contains supplementary material available at 10.1007/s40670-022-01558-1.

Entrustment Decision Making in the Core Entrustable Professional Activities: Results of a Multi-Institutional Study.

PURPOSE: In 2014, the Association of American Medical Colleges defined 13 Core Entrustable Professional Activities (EPAs) that all graduating students should be ready to do with indirect supervision upon entering residency and commissioned a 10-school, 5-year pilot to test implementing the Core EPAs framework. In 2019, pilot schools convened trained entrustment groups (TEGs) to review assessment data and render theoretical summative entrustment decisions for class of 2019 graduates. Results were examined to determine the extent to which entrustment decisions could be made and the nature of these decisions. METHOD: For each EPA considered (4-13 per student), TEGs recorded an entrustment determination (ready, progressing but not yet ready, evidence against student progressing, could not make a decision); confidence in that determination (none, low, moderate, high); and the number of workplace-based assessments (WBAs) considered (0->15) per determination. These individual student-level data were de-identified and merged into a multischool database; chi-square analysis tested the significance of associations between variables. RESULTS: The 2,415 EPA-specific determinations (for 349 students by 4 participating schools) resulted in a decision of ready (n = 997/2,415; 41.3%), progressing but not yet ready (n = 558/2,415; 23.1%), or evidence against student progression (n = 175/2,415; 7.2%). No decision could be made for the remaining 28.4% (685/2,415), generally for lack of data. Entrustment determinations' distribution varied across EPAs (chi-square P < .001) and, for 10/13 EPAs, WBA availability was associated with making (vs not making) entrustment decisions (each chi-square P < .05). CONCLUSIONS: TEGs were able to make many decisions about readiness for indirect supervision; yet less than half of determinations resulted in a decision of readiness to perform this EPA with indirect supervision. More work is needed at the 10 schools to enable authentic summative entrustment in the Core EPAs framework.

Revisiting the Core Entrustable Professional Activities for Entering Residency.

The Core EPAs for Entering Residency Pilot project aimed to test the feasibility of implementing 13 entrustable professional activities (EPAs) at 10 U.S. medical schools and to gauge whether the use of the Core EPAs could improve graduates' performance early in residency. In this manuscript, the authors (members of the pilot institutions and Association of American Medical Colleges staff supporting the project evaluation) describe the schools' capacity to collect multimodal evidence about their students' performance in each of the Core EPAs and the ability of faculty committees to use those data to make decisions regarding learners' readiness for entrustment. In reviewing data for each of the Core EPAs, the authors reflected on how each activity performed as an EPA informed by how well it could be assessed and entrusted. For EPAs that did not perform well, the authors examined whether there are underlying practical and/or theoretical issues limiting its utility as a measure of student performance in medical school.

Developing an Entrustment Process: Insights from the AAMC CoreEPA Pilot.

One of the main goals of the CoreEPA pilot has been to determine the feasibility of developing a process to make summative entrustment decisions regarding entrustable professional activities (EPAs). Five years into the pilot, we report results of a research study we conducted to explore approaches to the entrustment process undertaken by our ten participating schools. We sought to identify the choices that participating schools made regarding the entrustment process and why these decisions were made. We are sharing these results, highlighting ongoing challenges that were identified with the intent of helping other medical schools that are moving toward EPA-based assessment. We conducted semi-structured interviews with representatives of all 10 medical schools in the CoreEPA pilot to understand their choices in designing the entrustment process. Additional information was obtained through follow-up communication to ensure completeness and accuracy of the findings. Several common themes are described. Our results indicate that, while approaches to the entrustment process vary considerably, all schools demonstrated consistent adherence to the guiding principles of the pilot. Several common barriers to the entrustment process emerged, and there was a consensus that more experience is needed with the process before consequential entrustment decisions can be made. The CoreEPA pilot schools continue to address challenges identified in implementing entrustment processes and making entrustment decisions for our students graduating in the Class of 2020.

Simulation 2.0: Integrating Basic Scientists and Clinicians in a Simulation Environment.

An increasing number of medical schools are implementing curricular changes that better integrate clinical and basic sciences throughout all four years of medical school. One of the most frequently cited reasons is to improve medical student clinical reasoning skills while simultaneously aiming to decrease the attrition of basic science knowledge. Multiple pedagogical strategies have been explored to achieve this goal. We have found that simulation is a viable medium to integrate basic science within standardized patient encounters for early medical students.

Progress on a New Kind of Progress Test: Assessing Medical Students' Clinical Skills.

PROBLEM: Progress testing of medical knowledge has advantages over traditional medical school examination strategies. However, little is known about its use in assessing medical students' clinical skills or their integration of clinical skills with necessary science knowledge. The authors previously reported on the feasibility of the Progress Clinical Skills Examination (PCSE), piloted with a group of early learners. This subsequent pilot test studied the exam's validity to determine whether the PCSE is sensitive to the growth in students' clinical skills across the four years of medical school. APPROACH: In 2014, 38 medical student volunteers (years 1-4) in the traditional 2 + 2 curriculum at Michigan State University College of Human Medicine participated in the eight-station PCSE. Faculty and standardized patients assessed students' clinical skills, and faculty assessed students' responses to postencounter necessary science questions. Students performed pretest self-assessment across multiple measures and completed a posttest evaluation of their PCSE experience. OUTCOMES: Student performance generally increased by year in medical school for communication, history-taking, and physical examination skills. Necessary science knowledge increased substantially from first-year to second-year students, with less change thereafter. Students felt the PCSE was a fair test of their clinical skills and provided an opportunity to demonstrate their understanding of the related necessary science. NEXT STEPS: The authors have been piloting a wider pool of cases. In 2016, they adopted the PCSE as part of the summative assessment strategy for the medical school's new integrated four-year curriculum. Continued assessment of student performance trajectories is planned.

Finding a Path to Entrustment in Undergraduate Medical Education: A Progress Report From the AAMC Core Entrustable Professional Activities for Entering Residency Entrustment Concept Group.

PROBLEM: To better prepare graduating medical students to transition to the professional responsibilities of residency, 10 medical schools are participating in an Association of American Medical Colleges pilot to evaluate the feasibility of explicitly teaching and assessing 13 Core Entrustable Professional Activities for Entering Residency. The authors focused on operationalizing the concept of entrustment as part of this process. APPROACH: Starting in 2014, the Entrustment Concept Group, with representatives from each of the pilot schools, guided the development of the structures and processes necessary for formal entrustment decisions associated with students' increased responsibilities at the start of residency. OUTCOMES: Guiding principles developed by the group recommend that formal, summative entrustment decisions in undergraduate medical education be made by a trained group, be based on longitudinal performance assessments from multiple assessors, and incorporate day-to-day entrustment judgments by workplace supervisors. Key to entrustment decisions is evidence that students know their limits (discernment), can be relied on to follow through (conscientiousness), and are forthcoming despite potential personal costs (truthfulness), in addition to having the requisite knowledge and skills. The group constructed a developmental framework for discernment, conscientiousness, and truthfulness to pilot a model for transparent entrustment decision making. NEXT STEPS: The pilot schools are studying a number of questions regarding the pathways to and decisions about entrustment. This work seeks to inform meaningful culture change in undergraduate medical education through a shared understanding of the assessment of trust and a shared trust in that assessment.

Progress testing 2.0: clinical skills meets necessary science.

INTRODUCTION: Progress testing has been widely used in medical schools to test scientific knowledge but has not been reported for assessing clinical skills. DEVELOPMENT: We designed a novel progress examination that included assessments of both clinical performance and underlying basic and social science knowledge. This Progress Clinical Skills Examination (PCSE) was given to 21 early medical students at the beginning and end of a 6-week pilot test of a new medical school curriculum. IMPLEMENTATION: This examination was feasible for early students, easy to map to curricular objectives, and easy to grade using a combination of assessment strategies. FUTURE DIRECTIONS: Use of a PCSE is feasible for early medical students. As medical schools integrate clinical experience with underlying knowledge, this type of examination holds promise. Further data are needed to validate this examination as an accurate measure of clinical performance and knowledge.

What are the implications of implementation science for medical education?

BACKGROUND: Derived from multiple disciplines and established in industries outside of medicine, Implementation Science (IS) seeks to move evidence-based approaches into widespread use to enable improved outcomes to be realized as quickly as possible by as many as possible. METHODS: This review highlights selected IS theories and models, chosen based on the experience of the authors, that could be used to plan and deliver medical education activities to help learners better implement and sustain new knowledge and skills in their work settings. RESULTS: IS models, theories and approaches can help medical educators promote and determine their success in achieving desired learner outcomes. We discuss the importance of incorporating IS into the training of individuals, teams, and organizations, and employing IS across the medical education continuum. Challenges and specific strategies for the application of IS in educational settings are also discussed. CONCLUSIONS: Utilizing IS in medical education can help us better achieve changes in competence, performance, and patient outcomes. IS should be incorporated into curricula across disciplines and across the continuum of medical education to facilitate implementation of learning. Educators should start by selecting, applying, and evaluating the teaching and patient care impact one or two IS strategies in their work.

Documenting clinical performance problems among medical students: feedback for learner remediation and curriculum enhancement.

INTRODUCTION: We operationalized the taxonomy developed by Hauer and colleagues describing common clinical performance problems. Faculty raters pilot tested the resulting worksheet by observing recordings of problematic simulated clinical encounters involving third-year medical students. This approach provided a framework for structured feedback to guide learner improvement and curricular enhancement. METHODS: Eighty-two problematic clinical encounters from M3 students who failed their clinical competency examination were independently rated by paired clinical faculty members to identify common problems related to the medical interview, physical examination, and professionalism. RESULTS: Eleven out of 26 target performance problems were present in 25% or more encounters. Overall, 37% had unsatisfactory medical interviews, with 'inadequate history to rule out other diagnoses' most prevalent (60%). Seventy percent failed because of physical examination deficiencies, with missing elements (69%) and inadequate data gathering (69%) most common. One-third of the students did not introduce themselves to their patients. Among students failing based on standardized patient (SP) ratings, 93% also failed to demonstrate competency based on the faculty ratings. CONCLUSIONS: Our review form allowed clinical faculty to validate pass/fail decisions based on standardized patient ratings. Detailed information about performance problems contributes to learner feedback and curricular enhancement to guide remediation planning and faculty development.

Better data for teachers, better data for learners, better patient care: college-wide assessment at Michigan State University's College of Human Medicine.

When our school organized the curriculum around a core set of medical student competencies in 2004, it was clear that more numerous and more varied student assessments were needed. To oversee a systematic approach to the assessment of medical student competencies, the Office of College-wide Assessment was established, led by the Associate Dean of College-wide Assessment. The mission of the Office is to 'facilitate the development of a seamless assessment system that drives a nimble, competency-based curriculum across the spectrum of our educational enterprise.' The Associate Dean coordinates educational initiatives, developing partnerships to solve common problems, and enhancing synergy within the College. The Office also works to establish data collection and feedback loops to guide rational intervention and continuous curricular improvement. Aside from feedback, implementing a systems approach to assessment provides a means for identifying performance gaps, promotes continuity from undergraduate medical education to practice, and offers a rationale for some assessments to be located outside of courses and clerkships. Assessment system design, data analysis, and feedback require leadership, a cooperative faculty team with medical education expertise, and institutional support. The guiding principle is 'Better Data for Teachers, Better Data for Learners, Better Patient Care.' Better data empowers faculty to become change agents, learners to create evidence-based improvement plans and increases accountability to our most important stakeholders, our patients.

An interdisciplinary infection control education intervention: necessary but not sufficient.

BACKGROUND: Patient care environments are struggling to eradicate health care associated infections and studies of undergraduate and graduate medical trainees have revealed significant gaps in their performance of proper hand hygiene and aseptic technique (HH/AT), suggesting the need for improved curriculum. High-reliability industries have provided a model using standardization of approach and interprofessional training, and both are particularly suited to the teaching and assessment of these life-saving skills. The Infection Control Education project is a grant-funded, multi-institutional pilot launched to improve the teaching and assessment of HH/AT in our community. METHODS: An interprofessional team of leaders and educators from 2 local hospital systems and 3 health colleges developed a 9-component "ICE PACK," which includes a unanimously endorsed, detailed HH/AT checklist. This teaching and assessment module was delivered to nursing/medical student and postgraduate year 1 resident/nurse intern pairs. Retention of checklist skills was retested 2 to 5 months after participation in the module. RESULTS: Learner pairs participating in the 2-hour module mastered the HH/AT checklist and rated the experience highly. Retention after several weeks was disappointingly low in 2 of the 3 participant groups. CONCLUSIONS: A community-wide HH/AT checklist was developed and an ICE PACK of materials created that is portable, standardizes the teaching and assessment of HH/AT skills, and is designed for interprofessional pairs of learners. Retention of checklist steps was disappointing in most of participant groups. Multiple, simultaneous strategies for improving compliance with infection control mandates appear necessary.