Prospective implementation of correction for guessing in oral and maxillofacial pathology multiple-choice examinations: did student performance improve?

ABSTRACT

A standard correction for random guessing on multiple-choice examinations was implemented prospectively in an Oral and Maxillofacial Pathology course for second-year dental students. The correction was a weighted scoring formula for points awarded for correct answers, incorrect answers, and unanswered questions such that the expected gain in the multiple-choice examination score due to random guessing was zero. An equally weighted combination of four examinations using equal numbers of short-answer questions and multiple-choice questions was used for student evaluation. Scores on both types of examinations, after implementation of the correction for guessing on the multiple-choice component (academic year 2005-06), were compared with the previous year (academic year 2004-05) when correction for guessing was not used for student evaluation but was investigated retrospectively. Academically, the two classes were comparable as indicated by the grade distributions in a General Pathology course taken immediately prior to the Oral and Maxillofacial Pathology course. Agreement between scores on short-answer examinations and multiple-choice examinations was improved in the 2005-06 class compared with the 2004-05 class. Importantly, the test score means were higher on both the short-answer and multiple-choice examinations in the Oral and Maxillofacial Pathology course, and the standard deviations were significantly smaller in 2005-06 compared to 2004-05; these differences reflected an upward shift in the lower part of the grade distributions to higher grades in 2005-06. Furthermore, when students were classified by their grade in the General Pathology course, students receiving a C (numerical grade of 70-79 percent) in General Pathology had significantly improved performance in the Oral and Maxillofacial Pathology course in 2005-06, relative to 2004-05, on both short-answer and multiple-choice examinations representing an aptitude-treatment interaction. We interpret this improved performance as a response to a higher expectation imposed on the 2005-06 students by the prospective implementation of correction for guessing.