Evaluation of adding item-response theory analysis for evaluation of the European Board of Ophthalmology Diploma examination.
Acta Ophthalmol
; 91(7): e573-7, 2013 Nov.
Article
em En
| MEDLINE
| ID: mdl-23927770
ABSTRACT
PURPOSE:
To investigate whether introduction of item-response theory (IRT) analysis, in parallel to the 'traditional' statistical analysis methods available for performance evaluation of multiple T/F items as used in the European Board of Ophthalmology Diploma (EBOD) examination, has proved beneficial, and secondly, to study whether the overall assessment performance of the current written part of EBOD is sufficiently high (KR-20≥ 0.90) to be kept as examination format in future EBOD editions.METHODS:
'Traditional' analysis methods for individual MCQ item performance comprise P-statistics, Rit-statistics and item discrimination, while overall reliability is evaluated through KR-20 for multiple T/F items. The additional set of statistical analysis methods for the evaluation of EBOD comprises mainly IRT analysis. These analysis techniques are used to monitor whether the introduction of negative marking for incorrect answers (since EBOD 2010) has a positive influence on the statistical performance of EBOD as a whole and its individual test items in particular.RESULTS:
Item-response theory analysis demonstrated that item performance parameters should not be evaluated individually, but should be related to one another. Before the introduction of negative marking, the overall EBOD reliability (KR-20) was good though with room for improvement (EBOD 2008 0.81; EBOD 2009 0.78). After the introduction of negative marking, the overall reliability of EBOD improved significantly (EBOD 2010 0.92; EBOD 20110.91; EBOD 2012 0.91).CONCLUSION:
Although many statistical performance parameters are available to evaluate individual items, our study demonstrates that the overall reliability assessment remains the only crucial parameter to be evaluated allowing comparison. While individual item performance analysis is worthwhile to undertake as secondary analysis, drawing final conclusions seems to be more difficult. Performance parameters need to be related, as shown by IRT analysis. Therefore, IRT analysis has proved beneficial for the statistical analysis of EBOD. Introduction of negative marking has led to a significant increase in the reliability (KR-20 > 0.90), indicating that the current examination format can be kept for future EBOD examinations.Palavras-chave
Texto completo:
1
Coleções:
01-internacional
Base de dados:
MEDLINE
Assunto principal:
Oftalmologia
/
Conselhos de Especialidade Profissional
/
Interpretação Estatística de Dados
/
Educação de Pós-Graduação em Medicina
/
Avaliação Educacional
Tipo de estudo:
Evaluation_studies
Limite:
Female
/
Humans
/
Male
País/Região como assunto:
Europa
Idioma:
En
Revista:
Acta Ophthalmol
Assunto da revista:
OFTALMOLOGIA
Ano de publicação:
2013
Tipo de documento:
Article
País de publicação:
ENGLAND
/
ESCOCIA
/
GB
/
GREAT BRITAIN
/
INGLATERRA
/
REINO UNIDO
/
SCOTLAND
/
UK
/
UNITED KINGDOM