Development and Evaluation of a Student-Initiated Test Preparation Program for the USMLE Step 1 Examination.

PROBLEM: Studies have documented performance on the United States Medical Licensing Examination® (USMLE) Step 1 exam as an important factor that residency program directors consider when deciding which applicants to interview and rank. Therefore, success on this exam, though only one aspect of applicant evaluation, is important in determining future career prospects for medical students. Unfortunately, mean test scores at the University of Illinois College of Medicine at Chicago (UIC) have historically been below the national average. INTERVENTION: This retrospective and quasi-experimental mixed-methods study describes the development, evaluation, and effects of a student-initiated USMLE Step 1 preparatory program at UIC. The program provided second year students with First Aid for the USMLE Step 1 at the beginning of the academic year, as well as a six month subscription to the USMLE World question bank midyear. In addition, optional peer review sessions covering basic sciences and organ systems were taught by high-performing upperclassmen. The goals of the program were to raise mean USMLE Step 1 exam scores and increase the percentage of students passing the exam on their first time. CONTEXT: The program premiered during the 2012-13 academic year. Data from this cohort as well as four others (N = 830; 2010-2014 examinees) were gathered. Performances between preintervention (2010-12 examinees) and postintervention (2013-14 examinees) cohorts of students were compared. Focus groups and interviews with staff and students were conducted, recorded, and analyzed to investigate the impact that the program had on student interactions and perceptions of the learning environment. OUTCOME: There was a significant difference in exam performance pre- versus postintervention, with average USMLE Step 1 scores improving by 8.82 points following the implementation of the student-initiated program, t(5.61) = 828, p < .001. The average first-attempt pass rate also increased significantly by 8%, χ2(1) = 23.13, p < .001. Taking age, sex, Medical College Admission Test® scores, and undergraduate grade point average into account, students who participated in the program scored 6.57 points higher than students who did not participate in the program (R2 = 0.3), F(5, 886) = 76.71, p < .01, and had higher odds of passing USMLE Step 1 (odds ratio = 3.08, SE = 1.07, p < .01). Students and staff commented on the sense of community and empowerment the program created as well as the unique student-driven nature of the program. LESSONS LEARNED: This study demonstrates the efficacy of a student-initiated curriculum and provides guidance for development and implementation of examination preparatory efforts at other institutions.

Clinical Assessment Specialists: A New Faculty Role to Improve Clerkship Assessment.

INTRODUCTION: Ambulatory clerkships, including longitudinal integrated clerkships (LICs), face challenges to assessment, including time pressure and clinical demands on preceptors. High-quality clinical assessment is critical to implementing competency-based medical education, generating valid grades, and supporting learning. This importance is further heightened with the new pass/fail scoring for US Medical Licensing Exam Step 1, discontinuation of US Medical Licensing Exam Step 2 Clinical Skills, and the growing concern for bias in assessment. METHODS: The Kaiser Permanente Bernard J. Tyson School of Medicine's LIC spans the first 2 years with 50 students per class. In 2021-2022, the authors created a new faculty role, the clinical assessment specialist (CASp). CASps are highly trained clinical teachers who directly observe clerkship students in the ambulatory setting, provide feedback, and complete competency-based assessment forms. RESULTS: CASps completed 186 assessments of first-year (Y1) LIC students and 333 assessments of second-year (Y2) LIC students. Y2 students achieved average higher milestones and were rated as requiring less supervision compared to Y1 students. Y1 students rated CASps more favorably than Y2 students. Preceptors rated the contribution of CASps similarly across both years. Clerkship directors described benefits including identification of at-risk students and value of augmenting preceptor assessments. DISCUSSION: The CASp role may offer an innovative way to generate valid assessment of student performance, offset clinical pressures faced by preceptors, identify at-risk students, and mitigate bias, especially in an LIC. Future studies may examine assessment validity, including use in summative assessment. CONCLUSION: CASps are an innovative approach to clinical clerkship assessment.

Developing an Entrustment Process: Insights from the AAMC CoreEPA Pilot.

One of the main goals of the CoreEPA pilot has been to determine the feasibility of developing a process to make summative entrustment decisions regarding entrustable professional activities (EPAs). Five years into the pilot, we report results of a research study we conducted to explore approaches to the entrustment process undertaken by our ten participating schools. We sought to identify the choices that participating schools made regarding the entrustment process and why these decisions were made. We are sharing these results, highlighting ongoing challenges that were identified with the intent of helping other medical schools that are moving toward EPA-based assessment. We conducted semi-structured interviews with representatives of all 10 medical schools in the CoreEPA pilot to understand their choices in designing the entrustment process. Additional information was obtained through follow-up communication to ensure completeness and accuracy of the findings. Several common themes are described. Our results indicate that, while approaches to the entrustment process vary considerably, all schools demonstrated consistent adherence to the guiding principles of the pilot. Several common barriers to the entrustment process emerged, and there was a consensus that more experience is needed with the process before consequential entrustment decisions can be made. The CoreEPA pilot schools continue to address challenges identified in implementing entrustment processes and making entrustment decisions for our students graduating in the Class of 2020.

Self-Directed and Lifelong Learning: the 5th "S" for TBL™ Application Exercises?

This paper describes a pilot of an innovation to TBL™ as a means to add a fifth "S", self-directed learning (SDL) and life-long learning (LLL), to the 4S application exercise framework. It is important for TBL™ to explicitly address SDL and LLL to enhance student learning and meet accreditation standards.