Validity evidence for USMLE examination cut scores: results of a large-scale survey.

ABSTRACT

PURPOSE: This research examined the credibility of the cut scores used to make pass/fail decisions on United States Medical Licensing Examination (USMLE) Step 1, Step 2 Clinical Knowledge, and Step 3. METHOD: Approximately 15,000 members of nine constituency groups were asked their opinions about (1) current initial and ultimate fail rates and (2) the highest acceptable, lowest acceptable, and optimal initial and ultimate fail rates. RESULTS: Initial fail rates were generally viewed as appropriate; more variability was associated with ultimate fail rates. Actual fail rates for each examination across recent years fell within the range that respondents considered acceptable. CONCLUSIONS: Results provide important evidence to support the appropriateness of the cut scores used to make classification decisions for USMLE examinations. This evidence is viewed as part of the overall validity argument for decisions based on USMLE scores.