Multiple sclerosis and extract of cannabis: results of the MUSEC trial.

OBJECTIVE: Multiple sclerosis (MS) is associated with chronic symptoms, including muscle stiffness, spasms, pain and insomnia. Here we report the results of the Multiple Sclerosis and Extract of Cannabis (MUSEC) study that aimed to substantiate the patient based findings of previous studies. PATIENTS AND METHODS: Patients with stable MS at 22 UK centres were randomised to oral cannabis extract (CE) (N=144) or placebo (N=135), stratified by centre, walking ability and use of antispastic medication. This double blind, placebo controlled, phase III study had a screening period, a 2 week dose titration phase from 5 mg to a maximum of 25 mg of tetrahydrocannabinol daily and a 10 week maintenance phase. The primary outcome measure was a category rating scale (CRS) measuring patient reported change in muscle stiffness from baseline. Further CRSs assessed body pain, spasms and sleep quality. Three validated MS specific patient reported outcome measures assessed aspects of spasticity, physical and psychological impact, and walking ability. RESULTS: The rate of relief from muscle stiffness after 12 weeks was almost twice as high with CE than with placebo (29.4% vs. 15.7%; OR 2.26; 95% CI 1.24 to 4.13; p=0.004, one sided). Similar results were found after 4 weeks and 8 weeks, and also for all further CRSs. Results from the MS scales supported these findings. CONCLUSION: The study met its primary objective to demonstrate the superiority of CE over placebo in the treatment of muscle stiffness in MS. This was supported by results for secondary efficacy variables. Adverse events in participants treated with CE were consistent with the known side effects of cannabinoids. No new safety concerns were observed. TRIAL REGISTRATION NUMBER: NCT00552604.

A tool to measure the attributes of receiving IV therapy in a home versus hospital setting: the Multiple Sclerosis Relapse Management Scale (MSRMS).

BACKGROUND: Intravenous steroids are routinely used to treat disabling relapses in multiple sclerosis (MS). Theoretically, the infusion could take place at home, rather than in hospital. Findings from other patient populations suggest that patients may find the experiences of home relapse management more desirable. However, formal comparison of these two settings, from the patients' point of view, was prevented by the lack of a clinical scale. We report the development of a rating scale to measure patient's experiences of relapse management that allowed this question to be answered confidently. METHODS: Scale development had three stages. First, in-depth interviews of 21 MS patients generated a conceptual model and pool of potential scale items. Second, these items were administered to 160 people with relapsing-remitting MS. Standard psychometric techniques were used to develop a scale. Third, the psychometric properties of the scale were evaluated in a randomised controlled trial of 138 patients whose relapses were managed either at home or hospital. RESULTS: A preliminary conceptual model with eight dimensions, and a pool of 154 items was generated. From this we developed the MS Relapse Management Scale (MSRMS), a 42-item with four subscales: access to care (6 items), coordination of care (11 items), information (7 items), interpersonal care (18 items). The MSRMS subscales satisfied most psychometric criteria but had notable floor effects. CONCLUSIONS: The MSRMS is a reliable and valid measure of patients' experiences of MS relapse management. The high floor effects suggest most respondents had positive care experiences. Results demonstrate that patients' experiences of relapse management can be measured, and that the MSRMS is a powerful tool for determining which services to develop, support and ultimately commission.

Effect sizes can be misleading: is it time to change the way we measure change?

OBJECTIVES: Previous comparisons of the ability to detect change in the Barthel Index (BI) and Functional Independence Measure motor scale (FIMm) have implied these two scales are equally responsive when examined using traditional effect size statistics. Clinically, this is counterintuitive as the FIMm has greater potential to detect change than the BI and raises concerns about the validity of effect size statistics as indicators of rating scale responsiveness. To examine these concerns, in this study a sophisticated psychometric analysis was applied, Rasch measurement to BI and FIMm data. METHODS: BI and FIMm data were examined from 976 people at a single neurorehabilitation unit. Rasch analysis was used to compare the responsiveness of the BI and FIMm at the group comparison level (effect sizes, relative efficiency, relative precision) and for each individual person in the sample by computing the significance of their change. RESULTS: Group level analyses from both interval measurements and ordinal scores implied the BI and FIMm had equivalent responsiveness (BI and FIMm effect size ranges -0.82 to -1.12 and -0.77 to -1.05, respectively). However, individual person level analyses indicated that the FIMm detected significant improvement in almost twice as many people as the BI (50%, n=496 vs 31%, n=298), and recorded less people as unchanged on discharge (FIMm=4%, n=38; BI=12%, n=115). This difference was found to be statistically significant (chi(2)=273.81; p<0.000). CONCLUSIONS: These findings demonstrate that effect size calculations are limited and potentially misleading indicators of rating scale responsiveness at the group comparison level. Rasch analysis at the individual person level showed the superior responsiveness of the FIMm, supporting clinical expectation, and its added value as a method for examining and comparing rating scale responsiveness.

The ADAS-cog in Alzheimer's disease clinical trials: psychometric evaluation of the sum and its parts.

BACKGROUND: The Alzheimer's Disease Assessment Scale Cognitive Behavior Section (ADAS-cog), a measure of cognitive performance, has been used widely in Alzheimer's disease trials. Its key role in clinical trials should be supported by evidence that it is both clinically meaningful and scientifically sound. Its conceptual and neuropsychological underpinnings are well-considered, but its performance as an instrument of measurement has received less attention. Objective To examine the traditional psychometric properties of the ADAS-cog in a large sample of people with Alzheimer's disease. METHODS: Data from three clinical trials of donepezil (Aricept) in mild-to-moderate Alzheimer's disease (n=1421; MMSE 10-26) were analysed at both the scale and component level. Five psychometric properties were examined using traditional psychometric methods. These methods of examination underpin upcoming Food and Drug Administration recommendations for patient rating scale evaluation. RESULTS: At the scale-level, criteria tested for data completeness, scaling assumptions (eg, component total correlations: 0.39-0.67), targeting (no floor or ceiling effects), reliability (eg, Cronbach's α: = 0.84; test-retest intraclass correlations: 0.93) and validity (correlation with MMSE: -0.63) were satisfied. At the component level, 7 of 11 ADAS-cog components had substantial ceiling effects (range 40-64%). CONCLUSIONS: Performance was satisfactory at the scale level, but most ADAS-cog components were too easy for many patients in this sample and did not reflect the expected depth and range of cognitive performance. The clinical implication of this finding is that the ADAS-cog's estimate of cognitive ability, and its potential ability to detect differences in cognitive performance under treatment, could be improved. However, because of the limitations of traditional psychometric methods, further evaluations would be desirable using additional rating scale analysis techniques to pinpoint specific improvements.

Patient-orientated longitudinal study of multiple sclerosis in south west England (The South West Impact of Multiple Sclerosis Project, SWIMS) 1: protocol and baseline characteristics of cohort.

BACKGROUND: There is a need for greater understanding of the impact of multiple sclerosis (MS) from the perspective of individuals with the condition. The South West Impact of MS Project (SWIMS) has been designed to improve understanding of disease impact using a patient-centred approach. The purpose is to (1) develop improved measurement instruments for clinical trials, (2) evaluate longitudinal performance of a variety of patient-reported outcome measures, (3) develop prognostic predictors for use in individualising drug treatment for patients, particularly early on in the disease course. METHODS: This is a patient-centred, prospective, longitudinal study of multiple sclerosis and clinically isolated syndrome (CIS) in south west England. The study area comprises two counties with a population of approximately 1.7 million and an estimated 1,800 cases of MS. Self-completion questionnaires are administered to participants every six months (for people with MS) or 12 months (CIS). Here we present descriptive statistics of the baseline data provided by 967 participants with MS. RESULTS: Seventy-five percent of those approached consented to participate. The male:female ratio was 1.00:3.01 (n = 967). Average (standard deviation) age at time of entry to SWIMS was 51.6 (11.5) years (n = 961) and median (interquartile range) time since first symptom was 13.3 (6.8 to 24.5) years (n = 934). Fatigue was the most commonly reported symptom, with 80% of participants experiencing fatigue at baseline. Although medication use for symptom control was common, there was little evidence of effectiveness, particularly for fatigue. Nineteen percent of participants were unable to classify their subtype of MS. When patient-reported subtype was compared to neurologist assessment for a sample of participants (n = 396), agreement in disease sub-type was achieved in 63% of cases. There were 836 relapses, reported by 931 participants, in the twelve months prior to baseline. Twenty-three percent of the relapsing-remitting group and 12% of the total sample were receiving disease-modifying therapy at baseline. CONCLUSIONS: Demographics of this sample were similar to published data for the UK. Overall, the results broadly reflect clinical experience in confirming high symptom prevalence, with relatively little complete symptom relief. Participants often had difficulty in defining MS relapses and their own MS type.

Friedreich's ataxia impact scale: a new measure striving to provide the flexibility required by today's studies.

To develop, using modern techniques of rating scale construction, a new patient reported rating scale for clinical studies of Friedreich's ataxia (FA) that strives to meet the measurement needs of different types of study. Qualitative research methods were used to generate a conceptual framework of the impact of FA, and a pool of items necessary to construct a subscale for each area identified. This item pool was then administered to 492 people with FA. Rasch measurement methods guided scale construction. Eight areas for measurement were identified (speech, upper limb functioning, lower limb functioning, body movement, complex tasks, isolation, mood, self perceptions), and a 126-item scale constructed (FA Impact Scale; FAIS). In addition, three shorter versions were developed to illustrate how the FAIS can be adapted for observational studies, more disabled, and less disabled samples of people with FA. The FAIS is a psychometrically sound 126-item measure from which subsets of items can be selected to meet the needs of different studies. Importantly, all versions can be referred back to the original scale. This study shows one of the many clinical advantages of using Rasch measurement methods to construct rating scales.

The size of the treatment effect: do patients and proxies agree?

BACKGROUND: This study examined whether MS patients and proxy respondents agreed on change in disease impact, which was induced by treatment. This may be of interest in situations when patients suffer from limitations that interfere with reliable self-assessment, such as cognitive impairment. METHODS: MS patients and proxies completed the Multiple Sclerosis Impact Scale (MSIS-29) before and after intravenous steroid treatment. Analyses focused on patient-proxy agreement between MSIS-29 change scores. Transition ratings were used to measure the patient's judgement of change and whether this change was reflected in the MSIS-29 change of patients and proxies. Receiver operating characteristic (ROC) analyses were also performed to examine the diagnostic properties of the MSIS-29 when completed by patients and proxies. RESULTS: 42 patients and proxy respondents completed the MSIS-29 at baseline and follow-up. Patient-proxy differences between change scores on the physical and psychological MSIS-29 subscale were quite small, although large variability was found. The direction of mean change was in concordance with the transition ratings of the patients. Results of the ROC analyses of the MSIS-29 were similar when completed by patients (physical scale: AUC = 0.79, 95% CI: 0.65-0.93 and 0.66, 95% CI: 0.48-0.84 for the psychological scale) and proxies (physical scale: 0.80, 95% CI: 0.72-0.96 and 0.71, 95% CI: 0.56-0.87 for the psychological scale) CONCLUSION: Although the results need to be further explored in larger samples, these results do point towards possible use of proxy respondents to assess patient perceived treatment change at the group level.

The cervical dystonia impact profile (CDIP-58): can a Rasch developed patient reported outcome measure satisfy traditional psychometric criteria?

BACKGROUND: The United States Food and Drug Administration (FDA) are currently producing guidelines for the scientific adequacy of patient reported outcome measures (PROMs) in clinical trials, which will have implications for the selection of scales used in future clinical trials. In this study, we examine how the Cervical Dystonia Impact Profile (CDIP-58), a rigorous Rasch measurement developed neurologic PROM, stands up to traditional psychometric criteria for three reasons: 1) provide traditional psychometric evidence for the CDIP-58 in line with proposed FDA guidelines; 2) enable researchers and clinicians to compare it with existing dystonia PROMs; and 3) help researchers and clinicians bridge the knowledge gap between old and new methods of reliability and validity testing. METHODS: We evaluated traditional psychometric properties of data quality, scaling assumptions, targeting, reliability and validity in a group of 391 people with CD. The main outcome measures used were the CDIP-58, Medical Outcome Study Short Form-36, the 28-item General Health Questionnaire, and Hospital and Anxiety and Depression Scale. RESULTS: A total of 391 people returned completed questionnaires (corrected response rate 87%). Analyses showed: 1) data quality was high (low missing data < or = 4%, subscale scores could be computed for > 96% of the sample); 2) item groupings passed tests for scaling assumptions; 3) good targeting (except for the Sleep subscale, ceiling effect = 27%); 4) good reliability (Cronbach's alpha > or = 0.92, test-retest intraclass correlations > or = 0.83); and 5) validity was supported. CONCLUSION: This study has shown that new psychometric methods can produce a PROM that stands up to traditional criteria and supports the clinical advantages of Rasch analysis.

Rating scales as outcome measures for clinical trials in neurology: problems, solutions, and recommendations.

Have state-of-the-art clinical trials failed to deliver treatments for neurodegenerative diseases because of shortcomings in the rating scales used? This Review assesses two methodological limitations of rating scales that might help to answer this question. First, the numbers generated by most rating scales do not satisfy the criteria for rigorous measurements. Second, we do not really know which variables most rating scales measure. We use clinical examples to highlight concerns about the limitations of rating scales, examine their underlying rationales, clarify their implications, explore potential solutions, and make some recommendations for future research. We show that improvements in the scientific rigour of rating scales can improve the chances of reaching the correct conclusions about the effectiveness of treatments.

Quality of life measurement after stroke: uses and abuses of the SF-36.

BACKGROUND AND PURPOSE: The Medical Outcomes Study 36-item Short-Form Health Survey (SF-36) is widely used to measure health status after stroke. However, a fundamental assumption for its valid use after stroke has not been comprehensively tested: is it legitimate to generate scores for 8 scales and 2 summary measures using the standard algorithms? We tested this assumption. METHODS: SF-36 data from 177 people after stroke were examined (71% male; mean age, 62). We tested 6 scaling criteria to determine the legitimacy of generating the 8 SF-36 scale scores using Likert's method of summed ratings, and we tested 2 scaling criteria to determine the appropriateness of the standard SF-36 algorithms for weighting and combining scale scores to generate 2 summary measures (physical and mental). RESULTS: Scaling assumptions were fully satisfied for 6 of the 8 scales, but 3 of these 6 scales had notable floor and/or ceiling effects. Assumptions for generating 2 SF-36 summary measures were not satisfied. CONCLUSIONS: In this sample, 5 of the 8 SF-36 scales had limited validity as outcome measures after stroke, and the reporting of physical and mental summary scores was not supported. Results raise questions about the use of the SF-36 in stroke, and the SF-12 that is developed from it, and highlight the importance of testing scaling assumptions when applying existing scales to new populations.

Socio-demographic variables are limited predictors of health status in multiple sclerosis.

BACKGROUND AND OBJECTIVES: Self-reported health status measures reflect disease impact from the patient's perspective. However, such measures are not designed for individual patient use and are rarely used to guide clinical practice. Nevertheless, if strong predictors of health status can be demonstrated in large datasets, these could be used to identify people at risk of poor health states and help target interventions. The aim of this study was to examine the predictive value of routinely collected socio-demographic variables on health status. METHOD: Data for 638 patients with multiple sclerosis (MS) on the eight health dimensions of the Medical Outcomes Study 36-item Short Form Health Survey (SF-36) were collected either by a postal survey or hospital attendance and analysed by multiple regression analyses. RESULTS: Several sociodemographic variables, such as unemployment and manual social class had some predictive value on health status, but the effect was not strong (maximum cumulative variance explained 53 %). CONCLUSIONS: Sociodemographic variables that we studied were limited predictors of health status in MS and are of limited value in guiding clinical practice.

What sample sizes for reliability and validity studies in neurology?

Rating scales are increasingly used in neurologic research and trials. A key question relating to their use across the range of neurologic diseases, both common and rare, is what sample sizes provide meaningful estimates of reliability and validity. Here, we address two questions: (1) to what extent does sample size influence the stability of reliability and validity estimates; and (2) to what extent does sample size influence the inferences made from reliability and validity testing? We examined data from two studies. In Study 1, we retrospectively reduced the total sample randomly and nonrandomly by decrements of approximately 50 % to generate sub-samples from n = 713-20. In Study 2, we prospectively generated sub-samples from n = 20-320, by entry time into study. In all samples we estimated reliability (internal consistency, item total correlations, test-retest) and validity (within scale correlations, convergent and discriminant construct validity). Reliability estimates were stable in magnitude and interpretation in all sub-samples of both studies. Validity estimates were stable in samples of n ≥ 80, for 75 % of scales in samples of n = 40, and for 50 % of scales in samples of n = 20. In this study, sample sizes of a minimum of 20 for reliability and 80 for validity provided estimates highly representative of the main study samples. These findings should be considered provisional and more work is needed to determine if these estimates are generalisable, consistent, and useful.

The problem with health measurement.

In this review we discuss health measurement with a focus on psychometric methods and methodology. In particular, we examine some of the key issues currently facing the use of clinician and patient rating scales to measure the health outcomes of disease and treatment. We present three key facts and flag one crucial problem. First, the numbers generated by scales are increasingly used as the measurements of the central dependent variables upon which clinical decisions are frequently made. The rising profile of rating scales has significant implications for scale construction, evaluation, and selection, as well as for interpreting studies. Second, rating scale science is well established. Therefore, it is important to learn the lessons from those who have built and established the science over the last century. Finally, the goal of a rating scale is to measure. As such, over the last half century, developments in rating scale (psychometric) methods have caused a refocus in the way we should be measuring health. In particular, newer methods have significant clinical advantages over traditional approaches. These should be seriously considered for inclusion in everyday practice. This leads us to the central problem with health measurement, which is that we cannot currently be sure what most rating scales are measuring. This is because the methods we have in place to ensure the validity of rating scales fall short of what is actually required. We expand on this point, and provide some potential routes forward to help address this important problem.

Modafinil for multiple sclerosis fatigue: does it work?

UNLABELLED: Modafinil is sometimes used in the clinical setting for treating multiple sclerosis (MS) fatigue, despite conflicting and disappointing results from clinical trials. OBJECTIVE: We audited the patient perceived effectiveness of modafinil when used in clinical practice in a carefully selected group of patients with typical MS fatigue. METHODS: All MS patients commenced on modafinil for fatigue in the Oxford MS clinic were identified retrospectively, and the patient perceived benefit, side effects and decision on continuation of therapy at 1 month was obtained from the medical notes. Additionally all patients were subsequently interviewed, up to 5 years after treatment initiation. RESULTS: Surprisingly, 69% patients with fatigue also suffered excessive daytime sleepiness, and this subgroup found modafinil more beneficial. There was a better response against excessive daytime sleepiness than fatigue. The benefit against fatigue or sleepiness was generally sustained (median 1 year). CONCLUSION: This suggests that modafinil may be useful, particularly when MS fatigue is associated with sleepiness. The relationship between fatigue and excessive daytime sleepiness is poorly understood and needs to be better defined before appropriate measures of the most clinically relevant outcomes can be selected for clinical trials. Outcome measures used in previous clinical trials have not properly evaluated the effects of modafinil.

Evidence-based guidelines for using the Short Form 36 in cervical dystonia.

We aimed to provide evidence-based guidelines for using the Short Form 36 (SF-36) as an outcome measure in cervical dystonia (CD). To do this, we tested the hypothesized relationships between items, scales, and summary measures of the SF-36 using psychometric analyses in data from a postal survey of 235 people with CD. Although the majority of subscales performed adequately, the Role Physical and Role Emotional subscales had substantial floor and/or ceiling effects. Evidence did not support computing SF-36 Physical and Mental Component Summary scores. We propose guidelines that include the recommendation that these subscale and summary scores should be reported with caution.

Coordinating outcomes measurement in ataxia research: do some widely used generic rating scales tick the boxes?

The objective of this study was to examine the psychometric properties of four widely used generic health status measures in Friedreich's ataxia (FA), to determine their suitability as outcome measures. Fifty-six people with genetically confirmed FA completed the Barthel Index (BI), General Health Questionnaire (GHQ-12), EuroQol (EQ-5D), and Medical Outcomes Study 36-item Short Form Health Survey (SF-36) by means of postal survey. Six psychometric properties (data quality, scaling assumptions, acceptability, reliability, validity, and responsiveness) were examined. The response rate was 97%. In general, the psychometric properties of the four measures satisfied recommended criteria. However, closer examination highlighted limitations restricting their use for treatment trials. For example, the BI had high levels of missing data, EQ-5D had poor discriminant ability, and five SF-36 scales had high floor and/or ceiling effects. Most scale scores did not span the entire scale range, had means that differed notably from the scale mid-point, and had wide confidence intervals. Effect sizes (ES) were small for all four measures raising questions about their ability to detect clinically significant change. Results highlight the potential limitations of these four scales for evaluating health outcomes in FA and suggest the need for new disease-specific patient-based measures of its impact.

International Cooperative Ataxia Rating Scale (ICARS): appropriate for studies of Friedreich's ataxia?

Clinicians require scientifically rigorous, clinically meaningful rating scales to evaluate the health impact of disease and treatment that cannot be measured using conventional laboratory instruments. This study evaluated the psychometric properties of the International Cooperative Ataxia Rating Scale (ICARS), a commonly used clinician-rated measure, in Friedreich's ataxia (FRDA). People with confirmed FRDA were assessed by using the ICARS. Two assumptions of its measurement model were tested: the legitimacy of reporting ICARS scores in FRDA, and the acceptability, reliability, and validity of total and subscale scores. Seventy-seven people with FRDA were assessed. The ICARS total score effectively satisfied all psychometric criteria tested. The posture and gait disturbances subscale also performed well. The other three subscales did not pass standard criteria for tests of scaling assumptions, reliability, and validity. This small study recommends only the use of the ICARS total score as a measure of FRDA. However, the extent to which this score quantifies the true extent of FRDA remains uncertain as our validity testing was limited, partly by the lack of appropriate validating measures. Further validity testing, and examination of responsiveness, is required before the ICARS can be recommended as an outcome measure for treatment trials of FDRA.

Exploring rating scale responsiveness: does the total score reflect the sum of its parts?

Evaluating rehabilitation requires rating scales that detect change. The authors examined Barthel Index (BI) data from 1,495 patients at a neurorehabilitation unit to determine whether total scale responsiveness accurately reflects item responsiveness. Total score effect sizes were moderate to large (0.47 to 1.09). Item-level effect sizes (0.13 to 1.16) reveal floor (3.5 to 82.3%) and ceiling (9.7 to 95.4%) effects. Results suggest BI total score effect sizes may hide item-level weaknesses and may underestimate the impact of rehabilitation.

Patient-based outcomes of cervical dystonia: a review of rating scales.

Decisions on treatment choice for patients are based on trials and outcome studies that are wholly dependent upon the scientific quality of the rating scales used. This study reviewed rating scales used in cervical dystonia outcome research to determine the extent that they satisfy recommended criteria for rigorous measurement.