Standard setting for progress tests: combining external and internal standards.

ABSTRACT

OBJECTIVES: There has been little work on standard setting for progress tests and it is common practice to use normative standards. This study aimed to develop a new approach to standard setting for progress tests administered at the point when students approach graduation. METHODS: In this study we obtained performance data from newly qualified doctors and used this information to set the standard for the last progress test in the final year of undergraduate medical education. This external reference was validated against projections of student performance data based upon normative grading, and other published results. A simple linear growth model was used to set pass scores for progress tests earlier in the final year and this was also validated by published data. RESULTS: There was good agreement between standards set using the data from newly qualified doctors, the standard expected from extrapolation of the student progression data, and published performance data from another medical school. CONCLUSIONS: We have demonstrated that a combination of data from independent sources can be used to triangulate standard-setting decisions for progress tests. Performance data from successive cohorts of medical students could provide a fruitful source of information for standard setting for progress tests.