Development and Evaluation of a Student-Initiated Test Preparation Program for the USMLE Step 1 Examination.

PROBLEM: Studies have documented performance on the United States Medical Licensing Examination® (USMLE) Step 1 exam as an important factor that residency program directors consider when deciding which applicants to interview and rank. Therefore, success on this exam, though only one aspect of applicant evaluation, is important in determining future career prospects for medical students. Unfortunately, mean test scores at the University of Illinois College of Medicine at Chicago (UIC) have historically been below the national average. INTERVENTION: This retrospective and quasi-experimental mixed-methods study describes the development, evaluation, and effects of a student-initiated USMLE Step 1 preparatory program at UIC. The program provided second year students with First Aid for the USMLE Step 1 at the beginning of the academic year, as well as a six month subscription to the USMLE World question bank midyear. In addition, optional peer review sessions covering basic sciences and organ systems were taught by high-performing upperclassmen. The goals of the program were to raise mean USMLE Step 1 exam scores and increase the percentage of students passing the exam on their first time. CONTEXT: The program premiered during the 2012-13 academic year. Data from this cohort as well as four others (N = 830; 2010-2014 examinees) were gathered. Performances between preintervention (2010-12 examinees) and postintervention (2013-14 examinees) cohorts of students were compared. Focus groups and interviews with staff and students were conducted, recorded, and analyzed to investigate the impact that the program had on student interactions and perceptions of the learning environment. OUTCOME: There was a significant difference in exam performance pre- versus postintervention, with average USMLE Step 1 scores improving by 8.82 points following the implementation of the student-initiated program, t(5.61) = 828, p < .001. The average first-attempt pass rate also increased significantly by 8%, χ2(1) = 23.13, p < .001. Taking age, sex, Medical College Admission Test® scores, and undergraduate grade point average into account, students who participated in the program scored 6.57 points higher than students who did not participate in the program (R2 = 0.3), F(5, 886) = 76.71, p < .01, and had higher odds of passing USMLE Step 1 (odds ratio = 3.08, SE = 1.07, p < .01). Students and staff commented on the sense of community and empowerment the program created as well as the unique student-driven nature of the program. LESSONS LEARNED: This study demonstrates the efficacy of a student-initiated curriculum and provides guidance for development and implementation of examination preparatory efforts at other institutions.

A case study of teaching presence in virtual problem-based learning groups.

Interest in conducting problem-based learning (PBL) on-line has increased to meet student and physician schedules. Little research describes skills needed to facilitate PBL on-line. In this paper we studied teaching presence in asynchronous PBL groups. Two raters, with average inter-rater agreements of 0.80, used an existing code to measure teaching presence in 62 PBL case discussions facilitated by one instructor over five years. This instructor was selected because of consistently high teaching evaluations. Messages sent by the instructor in the on-line PBL discussion were coded into three categories: instructional design and organization, facilitating discourse and direct instruction. Instructional design indicators were most frequent averaging 22.5 (SD = 5.6)/discussion. Facilitating discourse and direct instruction were comparable, 19.5(SD = 7.4) and 19.5 (SD = 6.7), respectively. Messages and indicators of teacher presence rose across time with a decline during subsequent PBL cases with the same group.