Setting standards for performance tests:a pilot study of three-level Angoff method / 中华医学教育探索杂志

ABSTRACT

Dr. Rachel Yudkowsky et. al. have published the paper in Academic Medicine [2008,83 (10 Suppl):S13–S16] demonstrating a reformed Angoff method in which items were assigned to "Yes/No/Maybe" three-level categories for judges to estimate whether a borderline students would or would not accomplish each item,compared with "Yes/No" two-level categories. Five judges were provided three-level Angoff ratings for seven checklists used in a clini-cal skills exam for fourth-year students. 45% of 121 items had at least one "Maybe" rating. About 10% of all ratings were in the "Maybe" category. Case failure rates varied considerably depending on the simulated severity of ratings. Overall failure rates were not substantially impacted. The three-level Angoff retains the cognitive simplicity of the Yes/No Angoff while addressing the challenge of items midrange for the borderline candidate and avoids the potential bias.