Academic and professional career outcomes of medical school graduates who failed USMLE Step 1 on the first attempt.

This study sought to determine the academic and professional outcomes of medical school graduates who failed the United States Licensing Examination Step 1 on the first attempt. This retrospective cohort study was based on pooled data from 2,003 graduates of six Midwestern medical schools in the classes of 1997-2002. Demographic, academic, and career characteristics of graduates who failed Step 1 on the first attempt were compared to graduates who initially passed. Fifty medical school graduates (2.5 %) initially failed Step 1. Compared to graduates who initially passed Step 1, a higher proportion of graduates who initially failed Step 1 became primary care physicians (26/49 [53 %] vs. 766/1,870 [40.9 %]), were more likely at graduation to report intent to practice in underserved areas (28/50 [56 %] vs. 419/1,939 [ 21.6 %]), and more likely to take 5 or more years to graduate (11/50 [22.0 %] vs. 79/1,953 [4.0 %]). The relative risk of first attempt Step 1 failure for medical school graduates was 13.4 for African Americans, 7.4 for Latinos, 3.6 for matriculants >22 years of age, 3.2 for women, and 2.3 for first generation college graduates. The relative risk of not being specialty board certified for those graduates who initially failed Step 1 was 2.2. Our observations regarding characteristics of graduates in our study cohort who initially failed Step 1 can inform efforts by medical schools to identify and assist students who are at particular risk of failing Step 1.

Effects of comprehensive educational reforms on academic success in a diverse student body.

CONTEXT: Calls for medical curriculum reform and increased student diversity in the USA have seen mixed success: performance outcomes following curriculum revisions have been inconsistent and national matriculation of under-represented minority (URM) students has not met aspirations. Published innovations in curricula, academic support and pipeline programmes usually describe isolated interventions that fail to affect curriculum-level outcomes. METHODS: United States Medical Licensing Examination (USMLE) Step 1 performance and graduation rates were analysed for three classes of medical students before (matriculated 1995-1997, n=517) and after (matriculated 2003-2005, n=597) implementing broad-based reforms in our education system. The changes in pipeline recruitment and preparation programmes, instructional methods, assessment systems, academic support and board preparation were based on sound educational principles and best practices. RESULTS: Post-reform classes were diverse with respect to ethnicity (25.8% URM students), gender (51.8% female), and Medical College Admissions Test (MCAT) score (range 20-40; 24.1% scored ≤ 25). Mean±standard deviation MCAT scores were minimally changed (from 27.2±4.7 to 27.8±3.6). The Step 1 failure rate decreased by 69.3% and mean score increased by 14.0 points (effect size: d=0.67) overall. Improvements were greater among women (failure rate decreased by 78.9%, mean score increased by 15.6 points; d=0.76) and URM students (failure rate decreased by 76.5%, mean score increased by 14.6 points; d=0.74), especially African-American students (failure rate decreased by 93.6%, mean score increased by 20.8 points; d=1.12). Step 1 scores increased across the entire MCAT range. Four- and 5-year graduation rates increased by 7.1% and 5.8%, respectively. CONCLUSIONS: The effect sizes in these performance improvements surpassed those previously reported for isolated interventions in curriculum and student support. This success is likely to have resulted from the broad-based, mutually reinforcing nature of reforms in multiple components of the education system. The results suggest that a narrow reductionist view of educational programme reform is less likely to result in improved educational outcomes than a system perspective that addresses the coordinated functioning of multiple aspects of the academic enterprise.

Comprehensive changes in the learning environment: subsequent step 1 scores of academically at-risk students.

BACKGROUND: During the past 10 years at our institution, a number of changes have been instituted in the learning environment, including instructional techniques, assessment methods, academic support, and explicit board preparation. METHOD: The authors studied the Step 1 performance of students with MCAT scores of 20 to 25 in our former and current curricula. Effect sizes were calculated for score improvement using adjusted means from ANCOVA with covariates of MCAT and age. RESULTS: The overall effect size was 0.48, with larger effects seen for underrepresented minority students overall (d = 0.64) and African American students especially (d = 0.77), representing medium to large effects. Overall failure rates decreased by two thirds. CONCLUSIONS: Comprehensive changes in the learning environment were followed by substantial improvement in Step 1 performance among academically at-risk students.