Best Practice Recommendations for Electronic Patient-Reported Outcome Dataset Structure and Standardization to Support Drug Development.

The electronic patient-reported outcome (ePRO) Dataset Structure and Standardization Project is a multistakeholder initiative formed by Critical Path Institute's PRO Consortium and Electronic Clinical Outcome Assessment (eCOA) Consortium to address issues related to ePRO dataset structure and standardization and to provide best practice recommendations for clinical trial sponsors and eCOA providers. Given the many benefits of utilizing electronic modes to capture PRO data, clinical trials are increasingly using these methods, yet there are challenges to using data generated by eCOA systems. Clinical Data Interchange Standards Consortium (CDISC) standards are used in clinical trials to ensure consistency in data collection, tabulation, and analysis and to facilitate regulatory submission. Currently, ePRO data are not required to follow a standard model, and the data models used often vary by eCOA provider and sponsor. This lack of consistency creates risks for programming and analysis and difficulties for analytics functions generating the required analysis and submission datasets. There is a disconnect between data standards used for study data submission and those used for data collection via case report forms and ePRO forms, which would be mitigated through the application of CDISC standards for ePRO data capture and transfer. The project was formed to collate and examine the issues arising from the lack of adoption of standardized approaches and this paper details recommendations to address those issues. Recommendations to address issues with ePRO dataset structure and standardization include adopting CDISC standards in the ePRO data platform, timely involvement of key stakeholders, ensuring ePRO controls are implemented, addressing issues of missing data early in development, ensuring quality control and validation of ePRO datasets, and use of read-only datasets.

Psychometric Analysis of the Abdominal Score From the Diary for Irritable Bowel Syndrome Symptoms-Constipation Using Phase IIb Clinical Trial Data.

OBJECTIVES: The Diary for Irritable Bowel Syndrome Symptoms-Constipation (DIBSS-C) has been developed to assess the core signs and symptoms of irritable bowel syndrome with constipation (IBS-C). This article presents the psychometric evaluation of the DIBSS-C abdominal score. METHODS: Data for these analyses are from a multicenter phase IIb study in IBS-C patients (NCT02559206). Subjects completed a number of assessments via handheld electronic diary throughout the study. The analyses used the intent-to-treat population and were blinded to randomized treatment group. The analyses evaluated the reliability, validity, and responsiveness of the DIBSS-C abdominal score; identified an appropriate scoring algorithm; and determined thresholds for interpreting clinically meaningful changes at the individual level. RESULTS: The correlations between the DIBSS-C abdominal symptom items (ie, abdominal pain, discomfort, and bloating) were strong (>0.75). Cronbach's alpha for the abdominal symptom severity items was very strong (.94), indicating that the 3 abdominal symptom items produce a reliable score. The intraclass correlation coefficient for the abdominal score was 0.82, exceeding the threshold of 0.70 and indicating good test-retest reliability. Guyatt's responsiveness statistic values all exceeded the threshold for a large effect of 0.80, so the DIBSS-C abdominal score can be considered highly responsive to change. Triangulation across 3 sets of anchor-based analyses indicated that a threshold of -2.0 points on the abdominal score is an appropriate threshold for identifying meaningful change. CONCLUSIONS: Overall, this study provides evidence that the DIBSS-C abdominal score is valid, reliable, responsive to change, and interpretable for assessing treatment benefit in patients with IBS-C.

Moving from significance to real-world meaning: methods for interpreting change in clinical outcome assessment scores.

PURPOSE: Clinical outcome assessments (COAs) require evidence not only of reliability, validity, and ability to detect change, but also a definition of what constitutes a meaningful change on the instrument. The responder definition specifies the amount of change on the COA that may be interpreted as a treatment benefit and is critical for interpreting what constitutes a meaningful change on the COA scores. However, the literature that describes methods for developing and applying responder definitions can be difficult to navigate. Clear and concise guidelines regarding which methods to apply under what circumstances and how to interpret the results are lacking. This article provides a guide to the variety of available methods and issues that should be considered when establishing responder definitions for interpreting meaningful changes in COA scores. METHODS: An overview is provided for selecting anchors, developing study designs, planning psychometric analyses, using psychometric results to set responder thresholds, and applying responder thresholds in demonstrating treatment efficacy. RESULTS: There are a variety of anchor-based methods for consideration, but they all rely on a preference for strongly related and easily interpretable anchors. The benefits of applying multiple anchors and multiple analytic methods are discussed. The process of triangulation can synthesize results across multiple sources to gain confidence in a proposed responder definition. Though a link to meaningfulness from the patient's perspective is absent, distribution-based methods provide lower bound estimates of score precision and have a role in triangulation. Responder definitions are typically required within regulatory review, but their application may differ across clinical trial programs. CONCLUSIONS: By careful planning of anchor selection, study design, and psychometric methods, COA researchers can establish defensible responder thresholds that ultimately aid patients and clinicians in making informed treatment decisions.

Idio Scale Judgment: evaluation of a new method for estimating responder thresholds.

PURPOSE: To pilot the newly developed Idio Scale Judgment method for estimating the amount of score change that matters to patients (i.e., change thresholds). METHODS: An online panel of 500 participants diagnosed with multiple sclerosis (MS) responded to the Neuro-QoL fatigue scale and to demographic and clinical questions. Participants compared their own fatigue to that represented by seven short form summaries (SFSs) that were located relatively close to their own fatigue levels. They judged these as representing the same, greater, or less fatigue than their own. We calculated the distances between patients' own levels of fatigue and the location of SFSs they endorsed as a change that would make a difference in daily life. These distances were used as estimates of change thresholds. Logically inconsistent judgments were tabulated and associations with clinical and demographic variables were estimated. RESULTS: Change thresholds based on mean individual thresholds for consequential change were calculated for improvement (-3.5) and worsening (3.2). The majority of participants had no logically inconsistent judgments (69%). Having one or more reversals in judgment was not significantly associated with education or fatigue score, but was weakly associated with age, gender, and MS type and moderately associated with ratings of confidence in SFS comparisons. CONCLUSIONS: As piloted, Idio Scale Judgment had a number of design strengths. Participants made comparisons to levels of fatigue that were within range of their own, and their judgments were contextualized in personally relevant consequences. The design lends itself to collection of data in large samples allowing evaluation of the range of judgments. Some study limitations could be mitigated with modifications. We concluded that the Idio Scale Judgment has substantial promise as a new tool for estimating change thresholds.

Using classical test theory, item response theory, and Rasch measurement theory to evaluate patient-reported outcome measures: a comparison of worked examples.

OBJECTIVE: To provide comparisons and a worked example of item- and scale-level evaluations based on three psychometric methods used in patient-reported outcome development-classical test theory (CTT), item response theory (IRT), and Rasch measurement theory (RMT)-in an analysis of the National Eye Institute Visual Functioning Questionnaire (VFQ-25). METHODS: Baseline VFQ-25 data from 240 participants with diabetic macular edema from a randomized, double-masked, multicenter clinical trial were used to evaluate the VFQ at the total score level. CTT, RMT, and IRT evaluations were conducted, and results were assessed in a head-to-head comparison. RESULTS: Results were similar across the three methods, with IRT and RMT providing more detailed diagnostic information on how to improve the scale. CTT led to the identification of two problematic items that threaten the validity of the overall scale score, sets of redundant items, and skewed response categories. IRT and RMT additionally identified poor fit for one item, many locally dependent items, poor targeting, and disordering of over half the response categories. CONCLUSIONS: Selection of a psychometric approach depends on many factors. Researchers should justify their evaluation method and consider the intended audience. If the instrument is being developed for descriptive purposes and on a restricted budget, a cursory examination of the CTT-based psychometric properties may be all that is possible. In a high-stakes situation, such as the development of a patient-reported outcome instrument for consideration in pharmaceutical labeling, however, a thorough psychometric evaluation including IRT or RMT should be considered, with final item-level decisions made on the basis of both quantitative and qualitative results.

The development of a survey to measure completeness of response to migraine therapy.

OBJECTIVE: To assess the psychometric properties of a new patient-reported migraine instrument, the Completeness of Response Survey (CORS), which measures a comprehensive set of factors important to patients' decisions regarding the initiation and continuation of treatment. BACKGROUND: Traditionally, migraine treatments and the instruments used to demonstrate their efficacy have focused on the relief of headache pain. As new treatments emerge with the potential for more complete and consistent migraine relief, more comprehensive tools are needed to demonstrate these benefits. The CORS includes 2 modules, the static CORS, which comprehensively evaluates one treatment at one time point, and the comparative CORS, which provides a more global comparison between 2 treatments at one time point. Together, the 2 modules can measure unmet treatment needs and improvements over the course of a clinical study. METHODS: Data from an 8-site study comparing 147 patients' recent experiences with their current triptan therapy and 2 months of study treatment with a single-tablet formulation of sumatriptan/naproxen sodium were used to conduct a preliminary psychometric evaluation of the CORS. The study included both modules of the CORS, the Headache Impact Test, the revised Patient Perception of Migraine Questionnaire, and a migraine diary. RESULTS: The CORS response categories in both the static and comparative modules demonstrated limited floor or ceiling effects and few missing values (<3%). Inter-item correlations, principal components analysis (component loading range: 0.62 to 0.95), and high estimates of internal consistency (alpha range: 0.88 to 0.94) for each composite score supported the structure and proposed scoring algorithm for the static module. The pattern of correlations between the CORS static and comparative items and composites with the revised Patient Perception of Migraine Questionnaire items and subscales, as well as the relationships between responses to selected static CORS items and the migraine diary, supported the construct validity of the CORS. CONCLUSIONS: The CORS is capable of demonstrating advantages of more comprehensive migraine therapies over traditional therapies, which are primarily focused on the resolution of headache pain, by addressing the frequency and speed with which the most common migraine symptoms are resolved and patients' return to normal functioning. This research shows evidence for the value and utility for the CORS static and comparative items and components, and further evaluation is underway.

Sphygmomanometry-evoked allodynia in chronic pain patients with and without fibromyalgia.

BACKGROUND: Fibromyalgia is a chronic pain syndrome that affects about 2% of the U.S. general population, with greater prevalence among women (3.5%) than men (0.5%). Previous research results suggest that the experience of pain (allodynia) upon sphygmomanometry may indicate the presence of fibromyalgia. OBJECTIVE: The aim of this study was to confirm these findings in patients with fibromyalgia and other chronic pain conditions and evaluate the use of sphygmomanometry as a potential screening tool for the identification of patients with fibromyalgia. METHODS: A total of 150 people participated in this multicenter, cross-sectional observational study. The study included a physician-conducted evaluation to determine if the participant met the American College of Rheumatology (ACR) 1990 diagnostic criteria for fibromyalgia. The presence of sphygmomanometry-evoked allodynia was assessed during a seated cuff pressure inflation that was repeated three times on each arm. Each site was provided a sphygmomanometer to ensure standardization, and the pressure of the cuff at the moment of pain initiation was recorded. If the patient did not indicate pain prior to 180 mmHg, then the inflation was stopped, a notation of no pain was made, and a cuff pressure of 180 mmHg was recorded. The mean of the six cuff pressure measurements was used for the analyses. Logistic regression was performed to analyze the relationship between sphygmomanometry-evoked allodynia and fibromyalgia. RESULTS: The evaluable sample was 148 (one participant had too large an arm circumference for the sphygmomanometer and another did not receive the clinician evaluation of ACR-determined fibromyalgia diagnosis). Over half of the participants were determined to have an ACR diagnosis of fibromyalgia. Of these, 71 (91%) were women and had an average age of 54 years. Of the 70 participants with no fibromyalgia diagnosis, 42 (60%) were women and also had an average age of 54 years. Sixty-one (78%) of the fibromyalgia participants, compared with 25 (36%) of those with no fibromyalgia diagnosis, reported sphygmomanometry-evoked allodynia. The participants with fibromyalgia reported pain ata lower cuff pressure compared with those without fibromyalgia (132 mmHg vs. 166 mmHg, p < .01). The logistic regression showed that sphygmomanometry-evoked allodynia predicted an ACR-determined FM diagnosis (χ(2) = 19.4, p < .01). DISCUSSION: These findings support previous research suggesting that patients who report pain upon sphygmomanometry may warrant further evaluation for the presence of fibromyalgia.

The development and initial assessment of the strategy and leadership systems capability evaluation survey.

Hospital management and leadership systems are associated with organizational success and quality care. The Strategy and Leadership Systems Capability Evaluation (CE) survey was developed by GE Healthcare to assess management and leadership systems at health care institutions, serve as a benchmark for improvement, and measure progress. To assess the psychometric properties of the 29-item CE survey, including the factor structure, scoring algorithm, reliability, and discriminant validity, an online survey was completed by 3450 employees at 15 US hospitals. Of these employees, 609 worked at a hospital where a leadership and management intervention occurred after the initial survey administration. Data were also collected on job level, number of hospital beds, hospital ownership, location, community type, and the implementation of hospital interventions. Item response frequencies showed no floor or ceiling effects and limited missing data. Interitem correlations were strong without obvious redundancies, and factor analysis suggested a unidimensional scale. The resulting scale had strong internal consistency and was able to discriminate among known groups. The CE survey was developed to evaluate management and leadership systems at health care institutions. This study provides psychometric evidence in support of the reliability, validity, and scoring structure of this survey.

Interpreting Within-Patient Changes on the EORTC QLQ-C30 and EORTC QLQ-LC13.

INTRODUCTION: When determining if changes on patient-reported outcome (PRO) scores in clinical trials convey a meaningful treatment benefit, statistical significance tests alone may not communicate the patient perspective. Appraising within-patient changes on PRO scores against established thresholds can determine if improvements or deteriorations experienced by individuals are meaningful. To evaluate the appropriateness of thresholds for interpreting meaningful improvements and deterioration within individuals on the European Organisation for Research and Treatment of Cancer (EORTC) 30-item core instrument (QLQ-C30) and 13-item lung cancer module (QLQ-LC13), a series of psychometric methods were applied to data from a phase III randomized controlled clinical trial in non-small cell lung cancer. METHODS: Anchor-based methods of empirical cumulative distribution functions and classification statistics were employed using change scores from Baseline to Week 7 using changes on the QLQ-C30 Global Health Status item as an anchor. Distribution-based methods of one-half standard deviation and standard error of measurement identified the minimum amount of change each domain score can reliably measure. RESULTS: While the correlations between the domain scores and the anchor item were modest in size (i.e., r ≥ 0.30 for only 5 of 24 domains), consideration of multiple methods along with the magnitude of possible step changes on the score allowed for patterns to emerge. The triangulation process planned a priori resulted in different methods being the source for different domain scores. Absolute values of the proposed thresholds ranged from 11.11 to 33.33, and all resulted in the same classifications for all EORTC domains, except QLQ-C30 Fatigue, as would the 10-point threshold that is traditionally used. CONCLUSION: This study confirms the appropriateness of the 10-point EORTC score threshold generally used by the field for interpreting within-patient changes, but the thresholds proposed from this study enhance interpretability by corresponding to only observable locations along the domain score scale.

Asthma Daytime Symptom Diary (ADSD) and Asthma Nighttime Symptom Diary (ANSD): Measurement Properties of Novel Patient-Reported Symptom Measures.

BACKGROUND: The Asthma Daytime Symptom Diary (ADSD) and the Asthma Nighttime Symptom Diary (ANSD) were developed to meet the need for standardized patient-reported measures of asthma symptoms to assess treatment trial outcomes in adults and adolescents. OBJECTIVE: To determine scoring and evaluate the measurement properties of the ADSD/ANSD. METHODS: Adolescents (12-17 years) and adults (18+ years) with asthma completed draft 8-item electronic versions of the ADSD/ANSD for 10 days alongside the Adult Asthma Symptom Daily Scales (AASDS) and a Patient Global Impression of Severity (PGIS). Using classical and modern psychometric methods, initial analyses evaluated the performance of ADSD/ANSD items to inform scoring. Subsequent analyses evaluated the reliability and validity of ADSD/ANSD scores. RESULTS: A demographically and clinically diverse sample (n = 130 adolescents; n = 89 adults) was recruited. Item performance was generally strong. However, items assessing chest pressure and mucus/phlegm demonstrated redundancy and poorer performance and were removed. Principal-components analysis, confirmatory factor analysis, and item response theory supported combining items to form 6-item total ADSD/ANSD scores. Internal consistency (α = 0.94-0.95) and test-retest reliability (intraclass correlation coefficient = 0.86-0.95) were strong. Strong correlations (r = 0.72-0.80) were observed between ADSD scores and AASDS items assessing asthma symptom frequency, bother, and impact on activities. Significant differences (P < .001) in mean ADSD/ANSD scores were observed between groups categorized by asthma severity (PGIS), asthma control, inhaler use, nebulizer use, activity limitations, and nighttime awakenings. CONCLUSIONS: The ADSD/ANSD items and scores demonstrated strong reliability and validity. Implementation of the measures in interventional studies will enable the evaluation of responsiveness and meaningful within-patient change.