Evaluating the impact of calibration of patient-reported outcomes measures on results from randomized clinical trials: a simulation study based on Rasch measurement theory.

BACKGROUND: Meaningfully interpreting patient-reported outcomes (PRO) results from randomized clinical trials requires that the PRO scores obtained in the trial have the same meaning across patients and previous applications of the PRO instrument. Calibration of PRO instruments warrants this property. In the Rasch measurement theory (RMT) framework, calibration is performed by fixing the item parameter estimates when measuring the targeted concept for each individual of the trial. The item parameter estimates used for this purpose are typically obtained from a previous "calibration" study. But imposing this constraint on item parameters, instead of freely estimating them directly in the specific sample of the trial, may hamper the ability to detect a treatment effect. The objective of this simulation study was to explore the potential negative impact of calibration of PRO instruments that were developed using RMT on the comparison of results between treatment groups, using different analysis methods. METHODS: PRO results were simulated following a polytomous Rasch model, for a calibration and a trial sample. Scenarios included varying sample sizes, with instrument of varying number of items and modalities, and varying item parameters distributions. Different treatment effect sizes and distributions of the two patient samples were also explored. Cross-sectional comparison of treatment groups was performed using different methods based on a random effect Rasch model. Calibrated and non-calibrated approaches were compared based on type-I error, power, bias, and variance of the estimates for the difference between groups. RESULTS: There was no impact of the calibration approach on type-I error, power, bias, and dispersion of the estimates. Among other findings, mistargeting between the PRO instrument and patients from the trial sample (regarding the level of measured concept) resulted in a lower power and higher position bias than appropriate targeting. CONCLUSIONS: Calibration does not compromise the ability to accurately assess a treatment effect using a PRO instrument developed within the RMT paradigm in randomized clinical trials. Thus, given its essential role in producing interpretable results, calibration should always be performed when using a PRO instrument developed using RMT as an endpoint in a randomized clinical trial.

Does the Multiple Sclerosis Impact Scale-29 (MSIS-29) have the range to capture the experience of fully ambulatory multiple sclerosis patients? Learnings from the ASCLEPIOS studies.

Background: Trials of disease-modifying therapies (DMTs) for multiple sclerosis (MS) often include patients with minimal disability. Patient-reported outcome instruments used in these trials have often not captured physical and psychological treatment effects concomitant with observed clinical benefits. Objective: To examine whether the Multiple Sclerosis Impact Scale-29 (MSIS-29) captures changes in the impact of MS in a sample of patients enrolled in the Phase 3 ASCLEPIOS studies (ofatumumab vs. teriflunomide). Methods: Measurement properties (i.e. item fit, reliability, and targeting) of the MSIS-29 were analyzed using Rasch measurement theory (RMT) in data from two phase 3 ofatumumab clinical trials including patients with relapsing-remitting or secondary progressive MS (N = 1882). Targeting of the MSIS-29 items to the patient population was explored within groups categorized by Expanded Disability Status Scale (EDSS) scores. Results: Under RMT analyses, both the Physical and Psychological Impact scales of the MSIS-29 were not appropriately targeted to the overall sample of patients. In particular, 49% and 30% of patients with an EDSS score ≤ 2.5 had fewer physical and psychological impacts, respectively, than would typically be captured by these MSIS-29 items compared to patients with EDSS scores of ≥ 3. Conclusion: The MSIS-29 is commonly used to evaluate the patient-reported physical and psychological impact of MS. However, it may be limited in evaluating changes associated with DMTs in patients with minimal disability.

Establishing clinically meaningful within-individual improvement thresholds for eight patient-reported outcome measures in people with relapsing-remitting multiple sclerosis.

BACKGROUND: As disease-modifying therapies do not reverse the course of multiple sclerosis (MS), assessment of therapeutic success involves documenting patient-reported outcomes (PROs) concerning health-related quality of life, disease and treatment-related symptoms, and the impact of symptoms on function. Interpreting PRO data involves going beyond statistical significance to calculate within-patient meaningful change scores. These thresholds are needed for each PRO in order to fully interpret the PRO data. This analysis of PRO data from the PROMiS AUBAGIO study, which utilized 8 PRO instruments in teriflunomide-treated relapsing-remitting MS (RRMS) patients, was designed to estimate clinically meaningful within-individual improvement thresholds in the same manner, for 8 PRO instruments. RESULTS: The analytical approach followed a triangulation exercise that considered results from anchor- and distribution-based methods and graphical representations of empirical cumulative distribution functions in PRO scores in groups defined by anchor variables. Data from 8 PRO instruments (MSIS-29 v2, FSMC, MSPS, MSNQ, TSQM v1.4, PDDS, HRPQ-MS v2, and HADS) were assessed from 434 RRMS patients. For MSIS-29 v2, FSMC, MSPS, and MSNQ total scores, available anchor variables enabled both anchor- and distribution-based methods to be applied. For instruments with no appropriate anchor available, distribution-based methods were applied. A recommended value for meaningful within-individual improvement was defined by comparing mean change in PRO scores between participants showing improvement of one or two categories in the anchor variable or those showing no change. A "lower bound" estimate was calculated using distribution-based methods. An improvement greater than the lower-bound estimate was considered "clinically meaningful". CONCLUSION: This analysis produced estimates for assessing meaningful within-individual improvements for 8 PRO instruments used in MS studies. These estimates should be useful for interpreting scores and communicating study results and should facilitate decision-making by regulatory and healthcare authorities where these 8 PROs are commonly employed.

A toolbox of different approaches to analyze and present PRO-CTCAE data in oncology studies.

BACKGROUND: The patient-reported outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) is used to assess symptomatic adverse events in oncology trials. Currently, no standard for PRO-CTCAE analysis exists. METHODS: Key methods of descriptive analysis and longitudinal modeling using PRO-CTCAE data from an oncology clinical trial, DRiving Excellence in Approaches to Multiple Myeloma-2 (DREAMM-2), a phase II trial of belantamab mafodotin in multiple myeloma (NCT03525678), were explored. Descriptive methods included maximum postbaseline ratings, mean change over time, ratings above a predefined cutoff, line graphs, and stacked bar charts to illustrate patient-reported adverse events at one timepoint or dynamics over time. Analysis methods involving modeling over time included toxicity over time (ToxT) (repeated measurement model, time-to-event, area under the curve analyses), generalized estimating equations (GEE), and ordinal log-linear models (OLLMs). RESULTS: Visualizations of PRO-CTCAE data highlighted different aspects of the data. Selection of the appropriate visualization will depend on the audience and message to be conveyed. Consistent results were obtained by all modeling approaches; no difference was found between dose groups of the DREAMM-2 study in any PRO-CTCAE item by the ToxT approach or the more sophisticated GEE and OLLM methods. Interpretation of GEE results was the most challenging. OLLM supported the interval nature of the PRO-CTCAE response scale in the DREAMM-2 study. All modeling approaches account for multiple testing (driven by the number of items). CONCLUSIONS: Descriptive analyses and longitudinal modeling approaches are complementary approaches to presenting PRO-CTCAE data. In modeling, the ToxT approach may be a good compromise compared with more sophisticated analyses.

Is the St. George's Respiratory Questionnaire an Appropriate Measure of Symptom Severity and Activity Limitations for Clinical Trials in COPD? Analysis of Pooled Data from Five Randomized Clinical Trials.

Purpose: The objective of this study was to examine the psychometric properties of the St. George's Respiratory Questionnaire (SGRQ) in patients with chronic obstructive pulmonary disease (COPD) using Rasch measurement theory (RMT) analyses. Materials and Methods: RMT analysis was conducted on the baseline SGRQ data from five multi-national, Phase III randomized trials investigating a fixed-dose combination of a long-acting ß2-agonist and a long-acting muscarinic antagonist in COPD patients. Analysis was performed for the SGRQ "Symptoms" and "Activity" domains. An exploratory analysis was also conducted using the different specific symptoms as defined in the reconceptualization of the SGRQ "Symptoms" domain. Differential item functioning (DIF) analysis was performed for geographical regions on the "Activity" domain, in order to explore cross-cultural validity of the SGRQ. Results: Overall, the SGRQ "Activity" domain showed good measurement property, but two items ("Sitting or lying still making feel breathless" and "Playing sports or game making feel breathless") showed very high fit residuals. The SGRQ "Symptoms" domain demonstrated good targeting; however, two items showed disordered thresholds ("Coughed" and "Brought up phlegm"). In an exploratory RMT analysis, measures for "Cough and Sputum", "Breathing difficulties" or "Wheezing attacks" showed unsatisfactory measurement properties with poor reliability (person separation index = 0.35, 0.66 and 0.16, respectively) and targeting issues. The examination of cross-cultural performances of the SGRQ "Activity" items showed a great variability in the responses to these items in different global regions. Conclusion: Our results indicated that SGRQ may not be an appropriate instrument to measure symptom severity or activity limitations in patients with COPD. Hence, there is a need to develop other relevant PRO instruments that can be used in conjunction with SGRQ to provide a holistic assessment of the health status of COPD patients in clinical research.