Your browser doesn't support javascript.
loading
Cut-scores revisited: feasibility of a new method for group standard setting.
Shulruf, Boaz; Coombes, Lee; Damodaran, Arvin; Freeman, Adrian; Jones, Philip; Lieberman, Steve; Poole, Phillippa; Rhee, Joel; Wilkinson, Tim; Harris, Peter.
Affiliation
  • Shulruf B; University of New South Wales Australia, Sydney, Australia. b.shulruf@unsw.edu.au.
  • Coombes L; Cardiff University, Cardiff, UK.
  • Damodaran A; University of New South Wales Australia, Sydney, Australia.
  • Freeman A; University of Exeter, Exeter, UK.
  • Jones P; University of New South Wales Australia, Sydney, Australia.
  • Lieberman S; University of Texas, Austin, USA.
  • Poole P; University of Auckland, Auckland, New Zealand.
  • Rhee J; University of New South Wales Australia, Sydney, Australia.
  • Wilkinson T; University of Otago, Dunedin, New Zealand.
  • Harris P; University of New South Wales Australia, Sydney, Australia.
BMC Med Educ ; 18(1): 126, 2018 Jun 07.
Article in En | MEDLINE | ID: mdl-29879954
ABSTRACT

BACKGROUND:

Standard setting is one of the most contentious topics in educational measurement. Commonly-used methods all have well reported limitations. To date, there is not conclusive evidence suggesting which standard setting method yields the highest validity.

METHODS:

The method described and piloted in this study asked expert judges to estimate the scores on a real MCQ examination that they consider indicated a clear pass, clear fail, and pass mark for the examination as a whole. The mean and SD of the judges responses to these estimates, Z scores and confidence intervals were used to derive the cut-score and the confidence in it.

RESULTS:

In this example the new method's cut-score was higher than the judges' estimate. The method also yielded estimates of statistical error which determine the range of the acceptable cut-score and the estimated level of confidence one may have in the accuracy of that cut-score.

CONCLUSIONS:

This new standard-setting method offers some advances, and possibly advantages, in that the decisions being asked of judges are based on firmer constructs, and it takes into account variation among judges.
Subject(s)
Key words

Full text: 1 Collection: 01-internacional Database: MEDLINE Main subject: Students, Medical / Education, Medical / Educational Measurement Type of study: Prognostic_studies Limits: Humans Country/Region as subject: Oceania Language: En Journal: BMC Med Educ Journal subject: EDUCACAO Year: 2018 Document type: Article Affiliation country:

Full text: 1 Collection: 01-internacional Database: MEDLINE Main subject: Students, Medical / Education, Medical / Educational Measurement Type of study: Prognostic_studies Limits: Humans Country/Region as subject: Oceania Language: En Journal: BMC Med Educ Journal subject: EDUCACAO Year: 2018 Document type: Article Affiliation country:
...