Information collected during the residency match process does not predict clinical performance.

ABSTRACT

OBJECTIVE: To determine whether information collected during the National Resident Matching Program (NRMP) predicts clinical performance during residency. METHODS: Ten faculty members rated the overall quality of 69 pediatric house officers as clinicians. After rating by the faculty, folders were reviewed for absolute rank on the NRMP match list; relative ranking (where they ranked in their postgraduate year 1 [PGY-1] group); scores on part I of the National Board of Medical Examiners (NBME) examination; grades during medical school pediatrics and internal medicine rotations; membership in the Alpha Omega Alpha Medical Honor Society; scores of faculty interviews during intern application; scores on the pediatric in-service examination during PGY-1; and scores on the American Board of Pediatrics certification examination. RESULTS: There was substantial agreement among faculty raters as to the overall quality of the residents (agreement rate, 0.60; kappa = 0.50; P = .001). There was little correlation between faculty ratings and absolute (r = 0.19; P = .11) or relative (r = 0.20; P = .09) ranking on the NRMP match list. Individuals ranked in the top 10 of the match list had higher faculty ratings than did their peers (mean +/- SD, 3.66+/-1.22 vs. 3.0+/-1.27; P = .03), as did individuals ranked highest in their PGY-1 group (mean +/- SD, 3.88+/-1.45 vs. 3.04+/-1.24; P = .03). There was no correlation between faculty ratings and scores on part I of the NBME examination (r = 0.10; P = .49) or scores on the American Board of Pediatrics certification examination (r = 0.22; P = . 11). There were weak correlations between faculty ratings and scores of faculty interviews during the intern application process (r = 0.27; P = .02) and scores on the pediatric in-service examination during PGY-1 (r = 0.28; P = .02). There was no difference in faculty ratings of residents who were elected to Alpha Omega Alpha during medical school (mean +/- SD, 3.32+/-1.21) as compared with those who were not (mean +/- SD, 3.08+/-1.34) (P = .25). CONCLUSIONS: There is significant agreement among faculty raters about the clinical competence of pediatric residents. Medical school grades, performance on standardized examinations, interviews during the intern application process, and match-list ranking are not predictors of clinical performance during residency.