Evaluation of student learning in a problem-based curriculum - abstract

ABSTRACT

The Faculty of Health Services at McMaster University adopted, at its founding, continuous evaluation by peers and tutors as the primary tool to measure learning progress in its problem-based curriculum (PBL). Results were uneven and the student performance data showed poor reproducibility and little correlation with achievement in the final licensing examination (LMCC). Other tests were then introduced the "triple jump" exercise for tutorial learning and standardized patients for clinical skills, supplemented sporadically with short essays, free response items and patient management problems. These were costly and posed logistic problems, not justified by their psychometric properties. The objectives structured clinical examinations (OSCE) was added to test clinical skills and showed good results. But a reliable test of knowledge remained elusive. Having a fairly extensive experience with objective tests and knowing their positive psychometric properties, I introduced MCQs to my student groups in required and elective studies, in formative and summative fashion, to provide individual and group data. Administrators and unit supervisors, mostly unfamiliar with MCQs , disagreed with their use, asserting that they steered students to "cram for examinations" and were therefore anti-PBL! However, the higher LMCC failure rate of McMaster students compared with others persuaded the school to adopt an MCQ format based on one developed by the University of Limburg, Maastricht, Holland. The test is offered annually to all students, of all years, and samples the entire curriculum. When the three scores are obtained, a profile of performance emerges for each student. Results have shown higher percentages of correct answers in years 2 & 3, suggesting incremental learning. MCQs are a valued tool in evaluating knowledge. Future use should include extended choice and extended matching formats, which have been shown to improve the tests psychometrically (AU)