Effect of Standardised Scoring Conventions on Inter-rater Reliability in the Endoscopic Evaluation of Crohn's Disease.

ABSTRACT

BACKGROUND AND AIMS: The Crohn's Disease Endoscopic Index of Severity [CDEIS] and Simplified Endoscopic Score for Crohn's Disease [SES-CD] demonstrate consistent overall intra- and inter-rater reliability. However, the reliability of some index items is relatively poor. We evaluated scoring conventions to improve the reliability of these items. METHODS: Five gastroenterologists with no previous experience scoring the CDEIS or SES-CD were trained on their use. A total of 65 video recordings of colonoscopies were scored blindly by each gastroenterologist before and after additional training on index scoring conventions. Intra-class correlation coefficients [ICCs] assessed the effect of application of these conventions on the reliability of the CDEIS, SES-CD, and a Global Evaluation of Lesion Severity [GELS] score. RESULTS: Following training on scoring conventions, inter-rater ICCs (95% confidence interval [CI]) for the total SES-CD score increased from 0.78 [0.71, 0.85] to 0.85 [0.79, 0.89]. The ICCs for the total CDEIS and GELS scores were not affected corresponding inter-rater ICCs were 0.74 [0.65, 0.81] and 0.49, [0.38, 0.61] before and 0.73 [0.65, 0.81] and 0.53 [0.42, 0.64] following application of scoring conventions. Estimations of ulcer depth, surface area, anatomical location, and stenosis were important sources of variability. CONCLUSIONS: Use of scoring conventions previously developed by expert central readers enhanced the reliability of the SES-CD but did not similarly affect the CDEIS or GELS. As the SES-CD is more likely to be reliable than the CDEIS and can be optimised with targeted training, it is the preferred instrument for use in clinical trials.