Approaches to the Assessment of Clinical Benefit of Treatments for Conditions That Have Heterogeneous Symptoms and Impacts: Potential Applications in Rare Disease.

OBJECTIVES: Evaluating the clinical benefit of interventions for conditions with heterogeneous symptom and impact presentations is challenging. The same condition can present differently across and within individuals over time. This occurs frequently in rare diseases. The purpose of this review was to identify (1) assessment approaches used in clinical trials to address heterogeneous manifestations that could be relevant in rare disease research and (2) US Food and Drug Administration (FDA)-approved labeling claims that used these approaches. METHODS: A targeted literature review was conducted examining peer-reviewed publications and FDA-approved labeling claims from January 2002 to July 2020, focusing on claims incorporating clinical outcome assessments. Approaches were then assessed for their potential application in rare diseases. RESULTS: A total of 6 assessment approaches were identified: composite or other multicomponent endpoints, multidomain responder index, most bothersome symptom (MBS), goal attainment scaling, sliding dichotomy, and adequate relief. A total of 59 FDA-approved labeling claims associated with these approaches were identified: composite or other multicomponent endpoints (n=49), MBS (n=9), and adequate relief (n=1). A total of 10 FDA-approved labeling claims, all using multicomponent endpoints, were identified for rare diseases. CONCLUSIONS: Multicomponent, MBS, and adequate relief have been included in FDA-approved labeling claims. Multicomponent endpoints, including composite endpoints, were the most frequent way to address heterogeneous manifestations of both common and rare diseases. MBS may be acceptable to regulators, whereas multidomain responder index is unlikely to be. The goal attainment scaling and adequate relief approaches may have potential utility in rare disease trials, assuming the theoretical and statistical challenges inherent in each approach are managed.

Measuring respiratory symptoms of COPD: performance of the EXACT- Respiratory Symptoms Tool (E-RS) in three clinical trials.

BACKGROUND: Symptomatic relief is an important treatment goal for patients with COPD. To date, no diary for evaluating respiratory symptoms in clinical trials has been developed and scientifically-validated according to FDA and EMA guidelines. The EXACT - Respiratory Symptoms (E-RS) scale is a patient-reported outcome (PRO) measure designed to address this need. The E-RS utilizes 11 respiratory symptom items from the existing and validated 14-item EXACT, which measures symptoms of exacerbation. The E-RS total score quantifies respiratory symptom severity, and 3 domains assess breathlessness, cough and sputum, and chest symptoms. METHODS: This study examined the performance of the E-RS in each of 3 controlled trials with common and unique validation variables: one 6-month (N = 235, US) and two 3-month (N = 749; N = 597; international). Subjects completed the E-RS as part of a daily eDiary. Tests of reliability, validity, and responsiveness were conducted in each dataset. RESULTS: In each study, RS-Total score was internally consistent (Cronbach α) (0.88, 0.92, 0.92) and reproducible (intra-class correlation) in stable patients (2 days apart: 0.91; 7 days apart: 0.71, 0.74). RS-Total scores correlated significantly with the following criterion variables (Spearman's rho; p < 0.01, all comparisons listed here): FEV1% predicted (-0.19, -0.14, -0.15); St. George's Respiratory Questionnaire (SGRQ) (0.65, 0.52, 0.51); Breathlessness, Cough, and Sputum Scale (BCSS) (0.89, 0.89); modified Medical Research Council dyspnoea scale (mMRC) (0.40); rescue medication use (0.43, 0.42); Functional Performance Inventory Short-Form (FPI-SF) (0.43); 6-minute walk distance (6-MWT) (-0.30, -0.14) and incremental shuttle walk (ISWT) (-0.18) tests. Correlations between these variables and RS-Breathlessness, RS-Cough and Sputum, RS-Chest Symptoms scores supported subscale validity. RS-Total, RS-Breathlessness, and RS-Chest Symptoms differentiated mMRC levels of breathlessness severity (p < 0.0001). RS-Total and domain scores differentiated subjects with no rescue medication use and 3 or more puffs (p < 0.0001). Sensitivity to changes in health status (SGRQ), symptoms (BCSS), and exercise capacity (6MWT, ISWT) were also shown and responder definitions using criterion- and distribution-based methods are proposed. CONCLUSIONS: Results suggest the E-RS is a reliable, valid, and responsive measure of respiratory symptoms of COPD suitable for use in natural history studies and clinical trials. TRIAL REGISTRATION: MPEX: NCT00739648 ; AZ1: NCT00949975 ; AZ 2: NCT01023516.

The Fatigue Associated with Depression Questionnaire (FAsD): responsiveness and responder definition.

PURPOSE: The Fatigue Associated with Depression Questionnaire (FAsD) was developed to assess fatigue and its impact among patients with depression. The purpose of this study was to examine the questionnaire's responsiveness to change and identify a responder definition for interpretation of treatment-related changes. METHODS: Data were collected at baseline and at 6 weeks from patients with depression starting treatment with a new antidepressant. RESULTS: Of the 96 participants, 55.2% were women, with a mean age of 43.4 years. The total score and both subscales demonstrated statistically significant change with moderate to large effect sizes (absolute values ≥ 0.76). FAsD change scores were significantly correlated with change on the Brief Fatigue Inventory (r ≥ 0.73; p < 0.001). FAsD mean change scores discriminated among patient subgroups differing by degree of improvement in patient- and clinician-reported fatigue and depression. Responder definition for the two subscales and total score (0.67, 0.57, 0.62) was estimated primarily based on mean change among patients who reported a small but important improvement in fatigue. DISCUSSION: The FAsD was responsive to change, and the responder definition may be used when interpreting treatment-related change. Results add to previous findings suggesting the FAsD is a useful measure of fatigue among patients with depression.

Patient-reported outcome (PRO) measures for clinical trials of COPD: the EXACT and E-RS.

The precise assessment of treatment efficacy in clinical trials requires scientific instruments that are not only relevant to the target population and treatment, but have been shown to be reliable, valid, and sensitive to change within the intended context of use. This paper describes the background, procedures, and current status of 2 patient-reported outcome (PRO) instruments developed for use in clinical trials of chronic obstructive pulmonary disease (COPD). The first measure, the EXAcerbations of Chronic pulmonary disease Tool (EXACT), was developed under the EXACT-PRO Initiative, a multi-year, multi-sponsor project involving experts in pulmonary medicine, instrument development, and drug development regulatory issues, dedicated to the development of a single, standardized instrument for evaluating the effects of treatment on acute exacerbations of COPD. The second measure, the EXACT-Respiratory Symptoms (E-RS) scale, is a derivative instrument comprising a subset of EXACT items to test the effect of treatment on the severity of respiratory symptoms in stable COPD. The EXACT-PRO Initiative was the first PRO instrument development consortia, and the EXACT and E-RS are the first PRO measures to undergo qualification review by the United States Food and Drug Administration (FDA).

Measuring respiratory symptoms in moderate/severe asthma: evaluation of a respiratory symptom tool, the E-RS®: COPD in asthma populations.

BACKGROUND: Symptom constructs included in the Evaluating Respiratory Symptoms in Chronic Obstructive Pulmonary Disease (E-RS®: COPD) tool may be relevant to patients with asthma. The purpose of this study was to evaluate content validity and psychometric performance of the E-RS: COPD in moderate/severe asthma patients. METHODS: Content validity of the E-RS: COPD was evaluated in patients with moderate/severe asthma using concept elicitation and cognitive debriefing interviews. Secondary analyses using data from two clinical trials in patients with moderate/severe asthma evaluated the factor structure of the E-RS: COPD plus two supplementary items (wheeze; shortness of breath with strenuous physical activity) and assessed psychometric properties of the tool, which will be referred to as E-RS®: Asthma when used in asthma populations. RESULTS: Qualitative interviews (N = 25) achieved concept saturation for asthma respiratory symptoms. Concepts in the E-RS: COPD were relevant to patients and instructions were understood. Most patients (19/25; 76%) reported experiencing all concepts in the E-RS: COPD; no patients indicated missing symptoms. Secondary analyses of clinical trial data supported the original factor structure (RS-Total and three symptom-specific subscales). The two supplemental items did not fit with this factor structure and were not retained. RS-Total and subscale score reliability was high (internal consistency [α] > 0.70). Validity was demonstrated through significant (P < 0.0001) relationships with the St George's Respiratory Questionnaire (SGRQ) and Asthma Symptom Severity scale. E-RS: Asthma was responsive to change when evaluated using SGRQ, Patient Global Impression of Change and Asthma Quality of Life Questionnaire as anchors (P < 0.0001). Clinically meaningful change thresholds were also identified (RS-Total: - 2.0 units). CONCLUSIONS: The E-RS: Asthma is reliable and responsive for evaluating respiratory symptoms in patients with moderate/severe asthma.

Development of the neurotrophic keratopathy questionnaire: qualitative research.

BACKGROUND: Neurotrophic keratopathy/keratitis (NK) is a rare disease of the cornea that can lead to anatomical loss of the eye. Little is known about the NK experience from the patients' perspective. The objectives of this study were to examine the symptomatic experience and impacts of NK on patients and assess the overall comprehension, relevance, and content validity of a new questionnaire. METHODS: This was a cross-sectional, qualitative study conducted with NK patients with varying levels of disease severity, recruited from one clinical site. One-on-one interviews using concept elicitation and cognitive interviewing techniques were conducted. RESULTS: Fourteen NK patients participated; 64.3% were female (n = 9), mean age was 65.7 ± 13.3, and 14.3% (n = 2), 21.4% (n = 3), and 64.3% (n = 9) were classified as Mackie stage I, stage II, or stage III, respectively. Participants reported 24 concepts, including: redness (n = 12, 86%), sensitivity to light (n = 11, 79%), general discomfort (n = 9, 64%), dry eye (n = 9, 64%), reduced visual acuity (n = 9, 64%), blurred vision (n = 8, 57%), and eye fatigue (n = 8, 57%). No new concepts were reported after the 13th interview. The most frequently reported impacts included frustration (n = 10, 71%), driving impairment (n = 8, 57%), reading impairment (n = 7, 50%), difficulty watching television (n = 7, 50%), and concern with potentially losing their eyesight due to NK (n = 6, 43%). Participants provided positive feedback on the draft NK Questionnaire (NKQ) and felt that it was comprehensive and relevant to their experience with NK. Additionally, the recall period, instructions, item concepts, and response options were well-understood by participants. Minor revisions were made to the tool for consistency (i.e., the timeframe "in the past 7 days" was added to items 12-14); item 14 was modified to include "how often"; examples were added to item 9. CONCLUSIONS: The results of the concept elicitation portion of the qualitative study support the content validity of the draft NKQ. The clinically significant concepts identified in the literature and raised during concept elicitation are included as items in the questionnaire. Further assessment of the psychometric properties should be conducted in support of this new tool to measure the effect of new treatments on symptoms and impacts associated with NK.

The Short-term Impact of Symptom-defined COPD Exacerbation Recovery on Health Status and Lung Function.

Background: This study examined the short-term effects of symptom-defined exacerbation recovery on health status and pulmonary function in moderate to severe chronic obstructive pulmonary disease (COPD) patients. Methods: Secondary analyses of pooled data from two 12-week Phase II international, randomized controlled trials using the EXAcerbations of Chronic Pulmonary Disease Tool (EXACT) to identify symptom-defined exacerbations were conducted. Recovery was categorized as recovered, unrecovered (persistent worsening), or censored. Multiple regression analyses were used to test the effect of recovery status on change in the St George's Respiratory Questionnaire (SGRQ) and forced expiratory volume in 1 second (FEV1) from baseline to Week 12. Evaluating Respiratory Symptom scale (E-RS) scores were used to evaluate change in stable-state respiratory symptoms from baseline to Week 12. Results: Of 1346 eligible patients, 414 (31%) experienced ≥1 symptom-defined exacerbation; 260 patients recovered from their events, 80 experienced an unrecovered event (persistent worsening), 74 patients had only censored events (excluded). Groups were similar at baseline, with the recovered group reporting significantly worse symptoms (p<0.01). Recovery group and baseline SGRQ were significant predictors of change in health status over 12 weeks (p=0.04; p<0.01); no effects were observed for lung function. Significant between-group differences in change in respiratory symptom severity from baseline to Week 12 were observed (p<0.01), with the persistent worsening group experiencing clinically meaningful deterioration in breathlessness and chest symptoms. Conclusions: Results suggest some patients have difficulty recovering from symptom-defined exacerbations, leading to a deterioration in health status, dyspnea, and chest symptoms without short-term effects on lung function. Further study of symptom-defined exacerbation recovery and health outcomes is warranted.

Development of the Diabetes Injection Device Experience Questionnaire (DID-EQ) and Diabetes Injection Device Preference Questionnaire (DID-PQ).

BACKGROUND: Previous research has examined patient perceptions of insulin injection devices. However, injectable medications other than insulin are increasingly used to treat type 2 diabetes, including GLP-1 receptor agonists. No patient-reported outcome (PRO) instruments have been developed taking into account the experiences of patients using newer injection devices, which are often different from devices used for insulin. Therefore, the purpose of this qualitative study was to develop two draft PRO instruments focusing on patients' experiences with these newer injection devices (one instrument assessing perceptions of a single injection device, and another assessing preferences between two devices). METHODS: Questionnaire development proceeded in six steps: literature review, interviews with six device experts, concept elicitation interviews with patients (N = 32), preliminary translatability assessment, cognitive interviews with patients (N = 20), and final translatability assessment. RESULTS: Literature review and expert interviews were conducted to inform a concept elicitation interview guide. In concept elicitation in the US, UK, and Germany, patients with type 2 diabetes reported a range of injection features that influenced their perceptions of non-insulin injection devices (e.g., requirements for preparation of the medication/device, issues related to the needle, ease-of-use, portability). Two draft "item pools" were developed based on the literature review, expert interviews, and concept elicitation results. In cognitive interviews, patients recommended minor revisions and indicated that the draft instruments were generally clear, comprehensible, and relevant to their experience with non-insulin injectable medication. The instruments were refined based on the cognitive interviews and translatability assessment, resulting in two questionnaires. CONCLUSIONS: The various steps of qualitative research support the content validity of these new PRO instruments, which are the first developed specifically to assess perceptions of non-insulin injection delivery systems. Despite some overlap with insulin-focused questionnaires, the new instruments are distinct from previous instruments (omitting content that would not be relevant to patients receiving non-insulin injectable treatment, while including content that is not included in the insulin focused instruments). This qualitative research yielded two draft questionnaires that are grounded in patient perceptions and ready for psychometric validation studies with larger samples of patients with type 2 diabetes.

Qualitative evaluation of the St George's Respiratory Questionnaire in patients with severe asthma.

PURPOSE: Content validity is the extent to which a patient-reported outcome measure evaluates the concepts most relevant to a patient's condition and treatment. The St George's Respiratory Questionnaire (SGRQ) has been validated in a range of respiratory conditions. This study evaluated the content validity of the SGRQ in patients with severe asthma. METHODS: A qualitative study, guided by a protocol, which included concept elicitation and cognitive debriefing of the SGRQ, was conducted in patients aged ≥18 years with history of severe asthma and blood eosinophil counts of ≥150/µL (past month) or ≥300/µL (past 12 months). Patients were recruited until saturation for concept elicitation was achieved (i.e. no additional concepts identified). Concepts identified by the patients were then mapped to the SGRQ. RESULTS: 18 patient interviews provided concept saturation. Concept elicitation confirmed that the SGRQ includes the commonly reported asthma symptoms and their impact on daily life. In total, 89-100% of patients routinely experienced cough, nighttime awakenings, shortness of breath, chest tightness, sleep difficulty, phlegm/mucus, and wheezing. Patients reported asthma impacting daily and physical activities, mood and sleep. Cognitive interviewing confirmed that patients understood the instructions, items and response options in the SGRQ. Nearly half of the concepts in the SGRQ were endorsed by ≥12 patients; of the 17 items with scoring weights ≥85, 11 were mentioned by ≥12 patients. CONCLUSIONS: This study demonstrates that the SGRQ is a relevant, comprehensive and content-valid instrument to assess health status in patients with severe asthma.

Qualitative Research on Fatigue Associated with Depression: Content Validity of the Fatigue Associated with Depression Questionnaire (FAsD-V2).

BACKGROUND: Fatigue is one of the most common symptoms of major depressive disorder (MDD). The Fatigue Associated with Depression Questionnaire (FAsD) was developed to assess fatigue and its impact in patients with MDD. The current article presents the qualitative research conducted to develop and examine the content validity of the FAsD and FASD-Version 2 (FAsD-V2). METHODS: Three phases of qualitative research were conducted with patients recruited from a geographically diverse range of clinics in the US. Phase I included concept elicitation focus groups, followed by cognitive interviews. Phase II employed similar techniques in a more targeted sample. Phase III included cognitive interviews to examine whether minor edits made after Phase II altered comprehensibility of the instrument. Concept elicitation focused on patients' perceptions of fatigue and its impact. Cognitive interviews focused on comprehension, clarity, relevance, and comprehensiveness of the instrument. Data were collected using semi-structured discussion guides. Thematic analyses were conducted and saturation was examined. RESULTS: A total of 98 patients with MDD were included. Patients' statements during concept elicitation in phases I and II supported item development and content. Cognitive interviews supported the relevance of the instrument in the target population, and patients consistently demonstrated a good understanding of the instructions, items, response options, and recall period. Minor changes to instructions for the FAsD-V2 did not affect interpretation of the instrument. CONCLUSIONS: This qualitative research supports the content validity of the FAsD and FAsD-V2. These results add to previous quantitative psychometric analysis suggesting the FAsD-V2 is a useful tool for assessing fatigue and its impact in patients with MDD.

Identifying cases of undiagnosed, clinically significant COPD in primary care: qualitative insight from patients in the target population.

BACKGROUND: Many cases of chronic obstructive pulmonary disease (COPD) are diagnosed only after significant loss of lung function or during exacerbations. AIMS: This study is part of a multi-method approach to develop a new screening instrument for identifying undiagnosed, clinically significant COPD in primary care. METHODS: Subjects with varied histories of COPD diagnosis, risk factors and history of exacerbations were recruited through five US clinics (four pulmonary, one primary care). Phase I: Eight focus groups and six telephone interviews were conducted to elicit descriptions of risk factors for COPD, recent or historical acute respiratory events, and symptoms to inform the development of candidate items for the new questionnaire. Phase II: A new cohort of subjects participated in cognitive interviews to assess and modify candidate items. Two peak expiratory flow (PEF) devices (electronic, manual) were assessed for use in screening. RESULTS: Of 77 subjects, 50 participated in Phase I and 27 in Phase II. Six themes informed item development: exposure (smoking, second-hand smoke); health history (family history of lung problems, recurrent chest infections); recent history of respiratory events (clinic visits, hospitalisations); symptoms (respiratory, non-respiratory); impact (activity limitations); and attribution (age, obesity). PEF devices were rated easy to use; electronic values were significantly higher than manual (P<0.0001). Revisions were made to the draft items on the basis of cognitive interviews. CONCLUSIONS: Forty-eight candidate items are ready for quantitative testing to select the best, smallest set of questions that, together with PEF, can efficiently identify patients in need of diagnostic evaluation for clinically significant COPD.

Performance of the EXAcerbations of chronic pulmonary disease tool patient-reported outcome measure in three clinical trials of chronic obstructive pulmonary disease.

RATIONALE: The EXAcerbations of Chronic Pulmonary Disease Tool (EXACT) is a patient-reported outcome measure to standardize the symptomatic assessment of chronic obstructive pulmonary disease exacerbations, including reported and unreported events. The instrument has been validated in a short-term study of patients with acute exacerbation and stable disease; its performance in longer-term studies has not been assessed. OBJECTIVES: To test the EXACT's performance in three randomized controlled trials and describe the relationship between resource-defined medically treated exacerbations (MTEs) and symptom (EXACT)-defined events. METHODS: Prespecified secondary analyses of data from phase II randomized controlled trials testing new drugs for the management of chronic obstructive pulmonary disease: one 6-month trial (United States) (n = 235) and two 3-month, multinational trials (AZ 1 [n = 749], AZ 2 [n = 597]). In each case, the experimental drugs were found to be ineffective, permitting assessment of the EXACT's performance in three independent studies of moderate to severe high-risk patients on maintenance therapies. MEASUREMENTS AND MAIN RESULTS: The mean age of subjects was 62 to 64 years; 48 to 76% were male. Mean FEV1 % predicted was 42 to 59%. EXACT scores exhibited internal consistency (Cronbach's α ≥ 0.90), reproducibility (intraclass correlation ≥ 0.70), correlation with St. George's Respiratory Questionnaire (Spearman rho [rs] = 0.62, 0.46, 0.46 in the three trials; P < 0.001), and Breathlessness Cough and Sputum Scale (AZ 1, rs = 0.83; AZ 2, rs = 0.83; P < 0.001). EXACT-defined events had a high correspondence with alternative indicators of worsening (94, 88, and 93%). In each trial, unreported events were similar in severity (mean EXACT score, 56, 57, 61 vs. 53, 54 [P < 0.05], 57 [P < 0.05], respectively; 100-point scale) and longer (median, 9, 8, 7 vs. 8, 7 [P < 0.01], 6 days, respectively) than moderate MTEs. CONCLUSIONS: Data generated through the EXACT offers insight into the symptomatic nature of MTEs and the frequency, severity, and duration of unreported symptom-defined events. Clinical trials registered with www.clinicaltrials.gov (MPEX: NCT00739648; AZ 1: NCT00949975; AZ 2: NCT01023516).

Test-retest reliability of two patient-report measures for use in adults with ADHD.

AIMS: Childhood attention-deficit/hyperactivity disorder (ADHD) frequently persists into adulthood and continues to impair health-related quality of life (HRQL). Thus, it is important to have validated symptom and HRQL measures for assessing treatment outcomes in this population. The purpose of the current analysis was to assess test-retest reliability of two measures designed specifically for adults with ADHD: the Adult ADHD Self-Report Screener (ASRS) and the Adult ADHD Quality of Life Measure (AAQoL). METHODS: Data were collected at a US clinic specializing in the treatment of adult ADHD. Patients completed the ASRS (six-item ADHD symptom screening version) and the AAQoL (a 29-item condition-specific HRQL measure) at two visits, two weeks apart. At the second visit, patients also completed a measure of symptom stability so that test-retest reliability could be examined in the stable population. Test-retest reliability was examined through intraclass correlations (ICC) and t-tests comparing scores from the two visits. RESULTS: A total of 74 participants were enrolled (62.2% male; mean age = 38.6 years), and 43 of these participants attended both visits and were stable between visits (65.1% male; mean age = 39.3 years). The ICC assessing the association between Visit 1 and Visit 2 ASRS scores was 0.86, and the ICCs for the AAQoL subscales were 0.88 (Life Productivity), 0.75 (Psychological Health), 0.74 (Life Outlook), 0.78 (Relationships), and 0.86 (total score). The t-tests found no statistically significant differences between Visit 1 and Visit 2 scores for the ASRS or AAQoL. CONCLUSIONS: The ASRS and AAQoL demonstrated good test-retest reliability. Findings add to previous results suggesting that these instruments are useful outcome measures for treatments of ADHD in adults.