Do United States Medical Licensing Examination (USMLE) scores predict in-training test performance for emergency medicine residents?

BACKGROUND: Residency selection committees commonly utilize USMLE scores as criteria to screen residency applicants. OBJECTIVES: The purpose of this study is to evaluate the relationship between United States Medical Licensing Examination (USMLE) and American Board of Emergency Medicine (ABEM) in-training examination scores (ITEs). METHODS: In an Accreditation Council for Graduate Medical Education-accredited emergency medicine residency program, data were collected for this retrospective cohort study for the classes of 2002-2006. USMLE Step 1 and 2 scores and the ABEM ITEs were recorded for each post-graduate year (PGY) within the aforementioned time frame. Step 1 and 2 scores were compared to consecutive PGY ABEM ITEs to evaluate for an association. RESULTS: There were 51 USMLE Step 1 and 39 Step 2 scores available for comparison with 153 ABEM ITEs. The mean USMLE Step 1 and Step 2 scores were 228.9 (range 197-252) and 228.4 (range 168-259), respectively. The mean in-training percentiles for the PGY 1, 2, and 3 years were 40.4, 68.3, and 81.7, respectively. The R-squared values for the Step 1 scores compared to the PGY 1, 2, and 3 years' ITEs were 0.25, 0.18, and 0.16, respectively. The R-squared values for Step 2 scores as compared to the ABEM ITEs for the PGY 1, 2, and 3 years were 0.43, 0.44, and 0.38, respectively. Residents who scored below 200 on either USMLE Step 1 or Step 2 had significantly lower mean ABEM ITEs than residents who scored above 200 (p < 0.05) and were 10-fold more likely than residents who scored above 220 to score below the 70th percentile in their PGY3 ABEM ITE. CONCLUSIONS: USMLE Step 1 scores are mildly correlated and Step 2 scores are moderately correlated with ABEM ITEs. Scoring below 200 on either test is associated with significantly lower ABEM ITEs.