Should we still only rely on EDSS to evaluate disability in multiple sclerosis patients? A study of inter and intra rater reliability.

ABSTRACT

BACKGROUND: Few studies assessed reliability and inter-rater variability of EDSS and functional parameters (FP) rating. OBJECTIVE: To evaluate inter-rater variability and errors in EDSS and FP rating in junior (JN) and MS Neurologists (MSN). METHOD: Patients with MS were examined by a JN and a MSN on the same day. Each assessor rated FP and EDSS, then used a smartphone app to get an automated calculation for each FP ("smartphone" FP, sFP) and for EDSS ("smartphone" EDSS, sEDSS) from the description of the neurological exam. Inter-rater variability was assessed comparing JN and MSN ratings for each method. Intra-rater variability was assessed comparing traditional and digital rating for a given assessor. RESULT: 103 patients were included. Perfect agreement between JN and MSN was met for 67% and 70% of patients regarding EDSS and sEDSS. Disagreement that could lead to a significant difference in terms of level of disability occurred in 17% for EDSS and 12% for sEDSS (p=0.07). Regarding intra-rater reliability, we found 38 rating discrepancies for JN and 14 for MSN (p=0.04). CONCLUSION: We found a significant inter-rater variability as well as a substantial frequency of rating errors in JN. The use of less subjective, easier-to-rate scales should be encouraged.