Does surgical observed structured clinical exam (OSCE) predict clerkship grade, shelf exam scores, and preceptor clinical evaluation?

ABSTRACT

BACKGROUND: Clinical evaluation of medical student performance has been criticized as variable and subjective. The aim of this study was to assess the correlation of a summative surgical OSCE exam to clinical faculty evaluations as well as surgery shelf exam score and final grades. METHODS: The performance of 392 students who completed the surgical clerkship between 2017 and 2019 was assessed via Pearson Coefficients comparing OSCE grades, clinical evaluations of Medical Knowledge and Patient Care, Communication and Professionalism, the National Board of Medical Examiners (NBME) shelf surgical subject exam, and final clerkship grade. RESULTS: Results demonstrate a statistically significant positive relationship between the OSCE, Shelf score and grade, final clerkship grade, and all clinical evaluations except Communication skills. The greatest correlation occurred between OSCE and shelf scores and grades. Although significant, the degree of correlation with clinical observation was significantly less. CONCLUSION: This study demonstrates that a surgical OSCE has a small positive correlation with clinical knowledge as measured by the NBME shelf exam. There is also an equal correlation with medical knowledge standards, with the OSCE better predicting NBME shelf outcome. This lower correlation to clinical assessment suggests that either the clinical grades contain elements not detected on an OSCE exam but could also support the hypothesis that variability in clinical grades do contain a significant degree of subjectivity.