Reliability, Validity, and Responsiveness of InFLUenza Patient-Reported Outcome (FLU-PRO©) Scores in Influenza-Positive Patients.

OBJECTIVES: To assess the reliability, validity, and responsiveness of InFLUenza Patient-Reported Outcome (FLU-PRO©) scores for quantifying the presence and severity of influenza symptoms. METHODS: An observational prospective cohort study of adults (≥18 years) with influenza-like illness in the United States, the United Kingdom, Mexico, and South America was conducted. Participants completed the 37-item draft FLU-PRO daily for up to 14 days. Item-level and factor analyses were used to remove items and determine factor structure. Reliability of the final tool was estimated using Cronbach α and intraclass correlation coefficients (2-day reliability). Convergent and known-groups validity and responsiveness were assessed using global assessments of influenza severity and return to usual health. RESULTS: Of the 536 patients enrolled, 221 influenza-positive subjects comprised the analytical sample. The mean age of the patients was 40.7 years, 60.2% were women, and 59.7% were white. The final 32-item measure has six factors/domains (nose, throat, eyes, chest/respiratory, gastrointestinal, and body/systemic), with a higher order factor representing symptom severity overall (comparative fit index = 0.92; root mean square error of approximation = 0.06). Cronbach α was high (total = 0.92; domain range = 0.71-0.87); test-retest reliability (intraclass correlation coefficient, day 1-day 2) was 0.83 for total scores and 0.57 to 0.79 for domains. Day 1 FLU-PRO domain and total scores were moderately to highly correlated (≥0.30) with Patient Global Rating of Flu Severity (except nose and throat). Consistent with known-groups validity, scores differentiated severity groups on the basis of global rating (total: F = 57.2, P < 0.001; domains: F = 8.9-67.5, P < 0.001). Subjects reporting return to usual health showed significantly greater (P < 0.05) FLU-PRO score improvement by day 7 than did those who did not, suggesting score responsiveness. CONCLUSIONS: Results suggest that FLU-PRO scores are reliable, valid, and responsive to change in influenza-positive adults.

Beyond study participants: a framework for engaging patients in the selection or development of clinical outcome assessments for evaluating the benefits of treatment in medical product development.

BACKGROUND: Patients are participating more actively in health care decision-making with regard to their health, as well as in the broader realm of assessing the value of medical products and influencing decisions about their registration and reimbursement. There is an increasing trend to include patients' perspectives throughout the stages of medical product development by broadening the traditional study-participant role to that of an active partner throughout the process. Including patients in the selection and development of clinical outcome assessments (COAs) to evaluate the benefit of treatment is particularly important. Still, despite widespread enthusiasm, there is substantial uncertainty regarding how and when to engage patients in this process. PURPOSE: This manuscript proposes a methodological framework for engaging patients at varying levels in the selection and development of COAs for medical product development. FRAMEWORK: The framework builds on the Food and Drug Administration's roadmap for patient-focused COA. Methods for engaging patients across each stage in this roadmap are summarized by levels of engagement. Opportunities and examples of patient engagement (PE) in the selection and/or development of COAs are summarized, together with best practices and practical considerations. CONCLUSION: This paper offers a framework for understanding, planning, and implementing methods to advance PE in the selection and/or development of COAs for evaluating the benefit of medical products. The intent is to further this important discussion and enhance the process and outcome of PE in this context.

A New Approach for Identifying Patients with Undiagnosed Chronic Obstructive Pulmonary Disease.

RATIONALE: Chronic obstructive pulmonary disease (COPD) is often unrecognized and untreated. OBJECTIVES: To develop a method for identifying undiagnosed COPD requiring treatment with currently available therapies (FEV1 <60% predicted and/or exacerbation risk). METHODS: We conducted a multisite, cross-sectional, case-control study in U.S. pulmonary and primary care clinics that recruited subjects from primary care settings. Cases were patients with COPD and at least one exacerbation in the past year or FEV1 less than 60% of predicted without exacerbation in the past year. Control subjects were persons with no COPD or with mild COPD (FEV1 ≥60% predicted, no exacerbation in the past year). In random forests analyses, we identified the smallest set of questions plus peak expiratory flow (PEF) with optimal sensitivity (SN) and specificity (SP). MEASUREMENTS AND MAIN RESULTS: PEF and spirometry were recorded in 186 cases and 160 control subjects. The mean (SD) age of the sample population was 62.7 (10.1) years; 55% were female; 86% were white; and 16% had never smoked. The mean FEV1 percent predicted for cases was 42.5% (14.2%); for control subjects, it was 82.5% (15.7%). A five-item questionnaire, CAPTURE (COPD Assessment in Primary Care to Identify Undiagnosed Respiratory Disease and Exacerbation Risk), was used to assess exposure, breathing problems, tiring easily, and acute respiratory illnesses. CAPTURE exhibited an SN of 95.7% and an SP of 44.4% for differentiating cases from all control subjects, and an SN of 95.7% and an SP of 67.8% for differentiating cases from no-COPD control subjects. The PEF (males, <350 L/min; females, <250 L/min) SN and SP were 88.0% and 77.5%, respectively, for differentiating cases from all control subjects, and they were 88.0% and 90.8%, respectively, for distinguishing cases from no-COPD control subjects. The CAPTURE plus PEF exhibited improved SN and SP for all cases versus all control subjects (89.7% and 78.1%, respectively) and for all cases versus no-COPD control subjects (89.7% and 93.1%, respectively). CONCLUSIONS: CAPTURE with PEF can identify patients with COPD who would benefit from currently available therapy and require further diagnostic evaluation. Clinical trial registered with clinicaltrials.gov (NCT01880177).

Psychometric properties of the St George's Respiratory Questionnaire in patients with idiopathic pulmonary fibrosis.

The St George's Respiratory Questionnaire (SGRQ) has been used to measure health-related quality of life (HRQoL) in patients with idiopathic pulmonary fibrosis (IPF).This analysis evaluated the psychometric properties of the SGRQ using data from 428 patients with IPF who participated in a 12-month, randomised, placebo-controlled phase II trial of nintedanib.Internal consistency (Cronbach's α) was 0.91 for SGRQ total and >0.70 for domain scores. Test-retest reliability (intraclass correlation coefficients) was 0.77, 0.77, 0.69 and 0.66 for SGRQ total, activity, impact and symptoms scores, respectively. Construct validity of SGRQ total and domain scores was supported by weak to moderate cross-sectional correlations with the Medical Research Council dyspnoea scale (0.32-0.55), 6-min walk test distance (-0.25- -0.34), percentage predicted forced vital capacity (-0.11- -0.15) and measures of gas exchange (-0.26-0.03). There was some evidence that the SGRQ total score was sensitive to detecting change.The reliability, construct validity and responsiveness of the SGRQ in patients with IPF suggest that this is an acceptable measure of HRQoL in patients with IPF.

A systematic approach for studying the signs and symptoms of fever in adult patients: the fever assessment tool (FAST).

BACKGROUND: Although body temperature is one of four key vital signs routinely monitored and treated in clinical practice, relatively little is known about the symptoms associated with febrile states. The purpose of this study was to assess the validity, reliability and feasibility of the Fever Assessment Tool (FAST) in an acute care research setting. METHODS: Qualitative: To assess content validity and finalize the FAST instrument, 12 adults from an inpatient medical-surgical unit at the National Institutes of Health (NIH) Clinical Center participated in cognitive interviews within approximately 12 h of a febrile state (tympanic temperature ≥ 38° Celsius). Quantitative: To test reliability, validity and feasibility, 56 new adult inpatients completed the 21-item FAST. RESULTS: The cognitive interviews clarified and validated the content of the final 21-item FAST. Fifty-six patients completed the FAST from two to 133 times during routine vital sign assessment, yielding 1,699 temperature time points. Thirty-four percent of the patients (N = 19) experienced fever at one or more time points, with a total of 125 febrile time points. Kuder-Richardson 20 (KR-20) reliability of the FAST was 0.70. Four nonspecific symptom categories, Tired or Run-Down (12), Sleepy (13), Weak or Lacking Energy (11), and Thirsty (9) were among the most frequently reported symptoms in all participants. Using Generalized Estimating Equations (GEE), the odds of reporting eight symptoms, Warm (4), Sweating (5), Thirsty (9), General Body Aches (10), Weak or Lacking Energy (11), Tired or Run Down (12) and Difficulty Breathing (17), were increased when patients had a fever (Fever Now), compared to the two other subgroups-patients who had a fever, but not at that particular time point, (Fever Not Now) and patients who never had a fever (Fever Never). Many, but not all, of the comparisons were significant in both groups. CONCLUSION: Results suggest the FAST is reliable, valid and easy to administer. In addition to symptoms usually associated with fever (e.g. feeling warm), symptoms such as Difficulty Breathing (17) were identified with fever. Further study in a larger, more diverse patient population is warranted. TRIAL REGISTRATION: Clinical Trials Number: NCT01287143 (January 2011).

Development of the Flu-PRO: a patient-reported outcome (PRO) instrument to evaluate symptoms of influenza.

BACKGROUND: To develop content validity of a comprehensive patient-reported outcome (PRO) measure following current best scientific methodology to standardize assessment of influenza (flu) symptoms in clinical research. METHODS: Stage I (Concept Elicitation): 1:1 telephone interviews with influenza-positive adults (≥18 years) in the US and Mexico within 7 days of diagnosis. Participants described symptom type, character, severity, and duration. Content analysis identified themes and developed the draft Flu-PRO instrument. Stage II (Cognitive Interviewing): The Flu-PRO was administered to a unique set of influenza-positive adults within 14 days of diagnosis; telephone interviews addressed completeness, respondent interpretation of items and ease of use. RESULTS: Samples: Stage I: N = 46 adults (16 US, 30 Mexico); mean (SD) age: 38 (19), 39 (14) years; % female: 56%, 73%; race: 69% White, 97% Mestizo. Stage II: N = 34 adults (12 US, 22 Mexico); age: 37 (14), 39 (11) years; % female: 50%, 50%; race: 58% White, 100% Mestizo. SYMPTOMS: Symptoms identified by >50%: coughing, weak or tired, throat symptoms, congestion, headache, weakness, sweating, chills, general discomfort, runny nose, chest (trouble breathing), difficulty sleeping, and body aches or pains. No new content was uncovered during Stage II; participants easily understood the instrument. CONCLUSIONS: Results show the 37-item Flu-PRO is a content valid measure of influenza symptoms in adults with a confirmed diagnosis of influenza. Research is underway to evaluate the suitability of the instrument for children and adolescents. This work can form the basis for future quantitative tests of reliability, validity, and responsiveness to evaluate the measurement properties of Flu-PRO for use in clinical trials and epidemiology studies.

Development of the chronic obstructive pulmonary disease morning symptom diary (COPD-MSD).

BACKGROUND: The morning tends to be the most difficult time of day for many patients with chronic obstructive pulmonary disease (COPD) when symptoms can limit one's ability to perform even simple activities. Morning symptoms have been linked to higher levels of work absenteeism, thereby increasing the already substantial economic burden associated with COPD. A validated patient-reported outcome (PRO) instrument designed to capture morning symptoms will allow for a more comprehensive approach to the evaluation of treatment benefit in COPD clinical trials. METHODS: A qualitative interview study was conducted among a sample of symptomatic adults with COPD. Concept elicitation interviews (n = 35) were conducted to identify COPD morning symptoms, followed by cognitive interviews (n = 21) to ensure patient comprehension of the items, instructions and response options of the draft COPD Morning Symptom Diary (COPD-MSD). All interview transcript data were coded using ATLAS.ti software for content analysis. RESULTS: Mean age of the concept elicitation and cognitive interview sample was 65.0 years (±7.5) and 62.3 years (±8.3), respectively. The study sample represented the full range of COPD severity (Global Initiative for Chronic Lung Disease [GOLD] classifications I-IV) and included a mix of racial backgrounds, employment status and educational achievement. During the concept elicitation interviews, the three most frequently reported morning symptoms were shortness of breath (n = 35/35; 100 %), phlegm/mucus (n = 31/35; 88.6 %), and cough (n = 30/35; 85.7 %). A group of clinical and instrument development experts convened to review the concept elicitation data and develop the initial 32-item draft COPD-MSD. Cognitive interviews indicated subjects found the draft COPD-MSD to be comprehensive, clear, and easy to understand. The COPD-MSD underwent minor editorial revisions and streamlining based on cognitive interviews and input from the experts to yield the final 19-item daily diary. CONCLUSIONS: This study supports the content validity of the new COPD-MSD and positions the diary for quantitative psychometric testing.

Correlation of PROMIS scales and clinical measures among chronic obstructive pulmonary disease patients with and without exacerbations.

PURPOSE: The Patient-Reported Outcomes Measurement Information System (PROMIS(®)) initiative was developed to advance the methodology of PROs applicable to chronic diseases. Chronic obstructive pulmonary disease (COPD) is a progressive chronic disease associated with poor health. This study was designed to examine the correlation of PROMIS health-related quality of life (HRQOL) scales and clinical measures among COPD patients. METHODS: A cross-sectional analysis was conducted comparing patients who were stable (n = 100) with those currently experiencing a COPD exacerbation (n = 85). All PROMIS measures for adults available at the time of the study (2008), disease-targeted and other HRQOL instruments, health literacy, percent predicted FEV1, and a 6-min walk distance were assessed when patients were considered clinically stable. RESULTS: Stable COPD patients reported significantly (p ≤ 0.05) better health-related quality of life on PROMIS domains than patients experiencing an exacerbation. PROMIS domain scores were significantly (p ≤ 0.01) correlated with each of legacy measures. Six-min walk scores were most highly correlated with the PROMIS physical function domain scores (r = 0.53) followed by the fatigue (r = -0.26), social (r = 0.24) and to a lesser extent depression (r = -0.23), and anxiety (r = -0.22) domain scores. Percent predicted FEV1 score was significantly associated with PROMIS physical function scores (r = 0.27). CONCLUSION: This study provides support for the validity of the PROMIS measures in COPD patients.

Validation of Constant Work Rate Cycling Endurance Time for Use in Chronic Obstructive Pulmonary Disease Clinical Trials.

Rationale: A COPD Foundation working group sought to identify measures of exercise endurance, a meaningful aspect of physical functioning in everyday life among patients with chronic obstructive pulmonary disease (COPD) that is not fully accepted in regulatory decision making, hampering drug development. Objectives: To demonstrate, as we previously asserted (Casaburi COPD 2022;9:252), that constant work rate cycling endurance time is an appropriate exercise endurance measure in patients with COPD. Methods: To validate this assertion, we assembled an integrated database of endurance time responses, including 8 bronchodilator (2,166 subjects) and 15 exercise training (3,488 subjects) studies (Casaburi COPD 2022;9:520). Results: Construct validity was demonstrated: 1) peak physiologic and perceptual responses were similar for constant work rate and incremental cycling; 2) after bronchodilator therapy, there were greater increases in endurance time in patients with more severe airflow limitation; 3) after exercise training, endurance time increases were similar across airflow limitation severities; and 4) there were correlations between changes in endurance time and changes in mechanistically related physiologic and perceptual variables. Test-retest reliability was demonstrated, with consistency of changes in endurance time at two time points after the intervention. Responsiveness was confirmed, with significant increases in endurance time after active (but not placebo) bronchodilator therapy, with greater increases seen with more severe airflow limitation and after exercise training. On the basis of regression analysis using multiple anchor variables, the minimum important difference for endurance time increase is estimated to be approximately 1 minute. Conclusions: Constant work rate cycling endurance time is a valid exercise endurance measure in COPD, suitable for contributing to the evaluation of treatment benefit supporting regulatory decision making and evidence-based therapeutic recommendations.

Performance of the Cough and Sputum Assessment Questionnaire (CASA-Q) in COPD: Evidence from Clinical and Online Patient Interaction Studies.

Introduction: Patient perception of the burden of chronic bronchitis symptoms in chronic obstructive pulmonary disease (COPD) can be assessed using patient-reported outcome measures (PROMs). The Cough and Sputum Assessment Questionnaire (CASA-Q) was developed and tested for this purpose. This study reviewed the performance of the CASA-Q in published online studies and tested a novel approach to complement traditional methods of qualitative content validation. Methods: A targeted literature search was performed to identify published clinical studies of COPD using the CASA-Q as an endpoint. The performance of the questionnaire was examined in relation to other study endpoints, including clinical and functional measurements and other PROMs. Assessment of the content validity of the CASA-Q was carried out by comparing the content and structure of the questionnaire with published qualitative patient data from previously conducted online social media listening (SML) and online bulletin board (OBB) studies. Results: In the interventional clinical trials, CASA-Q change scores were consistent with study objectives and other endpoints, including FEV1 and other PROMs. Two observational studies showed cross-sectional correlations with other PROMs like the St.-George's Respiratory Questionnaire (SGRQ) and COPD assessment test (CAT) scores. Qualitative data from the SML and OBB patient studies were consistent with the content and structure of the CASA-Q, supporting the content validity of the measure. Conclusion: Results suggest that the CASA-Q is appropriately responsive to changes in cough and sputum symptoms and clinical impact in trials of COPD. The mapping of qualitative findings from online SML and OBB studies to CASA-Q domains and items confirm the content validity of the instrument. These results suggest the CASA-Q can be a valuable tool for evaluating treatment effect in COPD trials.

Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 1--eliciting concepts for a new PRO instrument.

The importance of content validity in developing patient reported outcomes (PRO) instruments is stressed by both the US Food and Drug Administration and the European Medicines Agency. Content validity is the extent to which an instrument measures the important aspects of concepts that developers or users purport it to assess. A PRO instrument measures the concepts most significant and relevant to a patient's condition and its treatment. For PRO instruments, items and domains as reflected in the scores of an instrument should be important to the target population and comprehensive with respect to patient concerns. Documentation of target population input in item generation, as well as evaluation of patient understanding through cognitive interviewing, can provide the evidence for content validity. Developing content for, and assessing respondent understanding of, newly developed PRO instruments for medical product evaluation will be discussed in this two-part ISPOR PRO Good Research Practices Task Force Report. Topics include the methods for generating items, documenting item development, coding of qualitative data from item generation, cognitive interviewing, and tracking item development through the various stages of research and preparing this tracking for submission to regulatory agencies. Part 1 covers elicitation of key concepts using qualitative focus groups and/or interviews to inform content and structure of a new PRO instrument. Part 2 covers the instrument development process, the assessment of patient understanding of the draft instrument using cognitive interviews and steps for instrument revision. The two parts are meant to be read together. They are intended to offer suggestions for good practices in planning, executing, and documenting qualitative studies that are used to support the content validity of PRO instruments to be used in medical product evaluation.

Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: part 2--assessing respondent understanding.

The importance of content validity in developing patient reported outcomes (PRO) instruments is stressed by both the US Food and Drug Administration and the European Medicines Agency. Content validity is the extent to which an instrument measures the important aspects of concepts developers or users purport it to assess. A PRO instrument measures the concepts most relevant and important to a patient's condition and its treatment. For PRO instruments, items and domains as reflected in the scores of an instrument should be important to the target population and comprehensive with respect to patient concerns. Documentation of target population input in item generation, as well as evaluation of patient understanding through cognitive interviewing, can provide the evidence for content validity. Part 1 of this task force report covers elicitation of key concepts using qualitative focus groups and/or interviews to inform content and structure of a new PRO instrument. Building on qualitative interviews and focus groups used to elicit concepts, cognitive interviews help developers craft items that can be understood by respondents in the target population and can ultimately confirm that the final instrument is appropriate, comprehensive, and understandable in the target population. Part 2 details: 1) the methods for conducting cognitive interviews that address patient understanding of items, instructions, and response options; and 2) the methods for tracking item development through the various stages of research and preparing this tracking for submission to regulatory agencies. The task force report's two parts are meant to be read together. They are intended to offer suggestions for good practice in planning, executing, and documenting qualitative studies that are used to support the content validity of PRO instruments to be used in medical product evaluation.

Validation of the Chinese version of the Functional Performance Inventory Short Form in patients with chronic obstructive pulmonary disease.

AIMS AND OBJECTIVE: To determine the reliability and validity of the Chinese version of the Functional Performance Inventory Short Form (FPI-SF-C) in patients with chronic obstructive pulmonary disease in Beijing, China. BACKGROUND: The Functional Performance Inventory Short Form (FPI-SF) is a 32-item instrument designed to measure self-reported functional performance of patients with chronic obstructive pulmonary disease. This instrument had not been translated into Mandarin Chinese and tested for use in mainland China. DESIGN: Cross-sectional validation study with a two-week test of reproducibility. METHOD: The FPI-SF was translated using forward and backward translation procedures and administered to 108 stable chronic obstructive pulmonary disease patients from outpatient clinics of three hospitals in Beijing, China. Pulmonary function and six-minute walking distance (6-MWD) tests were performed on the same day or within one week before the FPI-SF-C completion. Thirty of the patients participated in the reproducibility assessment. RESULTS: Cronbach's alpha for the FPI-SF-C total score was 0·89; subscales ranged from 0·70 (Body Care)-0·89 (Spiritual activities). Test-retest reliability (intraclass correlation coefficient) was 0·93 with subscales ranging from 0·73 (Body Care)-0·96 (Household Maintenance). No ceiling or floor effects were observed for total FPI-SF-C score. Total scores correlated significantly (p<0·05) with 6-MWD (r=0·56), modified Medical Research Council Dyspnoea Scale (MMRC) (r=-0·55), BODE index (r=-0·47), forced expiratory volume in one second (FEV(1)) (r=0·41) and FEV(1) % of predicted (r=0·26) and differentiated patients based on GOLD rating of COPD severity (χ(2)=16·22, p<0·001). CONCLUSIONS: Results suggest the FPI-SF-C is a reliable and valid instrument for measuring functional performance in mainland Chinese patients with chronic obstructive pulmonary disease. Further psychometric testing in a wide range of subjects and an evaluation of its utility in clinical practice are warranted. RELEVANCE TO PRACTICE: The FPI-SF-C may be useful for understanding difficulties in functional performance and evaluating the effect of treatment in Chinese patients with chronic obstructive pulmonary disease.

Development of the EXAcerbations of Chronic Obstructive Pulmonary Disease Tool (EXACT): a patient-reported outcome (PRO) measure.

BACKGROUND: This article describes the qualitative methods used to develop the EXAcerbation of Chronic Pulmonary Disease Tool (EXACT), a new patient-reported outcome (PRO) instrument for evaluating frequency, severity, and duration of exacerbations of chronic obstructive pulmonary disease (COPD). METHODS: Focus groups and interviews were conducted in the United States with COPD patients treated for exacerbations during the past 6 months. Participants were asked to describe exacerbation attributes, care-seeking cues, and indications of progression and recovery. An iterative process was used to identify themes in the data to inform instrument content and structure. Cognitive debriefing interviews were performed to evaluate and revise the draft item pool. Experts in COPD, instrument development, and clinical research participated in the process. RESULTS: Eighty-three subjects participated in elicitation focus groups or interviews (n=48); elicitation interviews with cognitive debriefing (n=23), or cognitive interviews alone (n=12). Mean age of the sample was 65 years (SD=10); 45% were male; mean FEV-1% predicted was 44% (SD=16). Participants characterized exacerbations as a persistent increase in the severity of respiratory symptoms and other systemic manifestations accompanied by a dramatic reduction in activity. Specific attributes included shortness of breath, chest congestion, cough, sputum, chest discomfort, feeling weak or tired, sleep disturbances, and concern or worry. The diary card of 23 candidate items was debriefed in booklet and electronic format. CONCLUSIONS: Qualitative data from patients and input from experts formed the basis of the EXACT's structure and item pool, ready for empirically based item reduction and reliability and validity testing.

Use of existing patient-reported outcome (PRO) instruments and their modification: the ISPOR Good Research Practices for Evaluating and Documenting Content Validity for the Use of Existing Instruments and Their Modification PRO Task Force Report.

BACKGROUND: Patient-reported outcome (PRO) instruments are used to evaluate the effect of medical products on how patients feel or function. This article presents the results of an ISPOR task force convened to address good clinical research practices for the use of existing or modified PRO instruments to support medical product labeling claims. The focus of the article is on content validity, with specific reference to existing or modified PRO instruments, because of the importance of content validity in selecting or modifying an existing PRO instrument and the lack of consensus in the research community regarding best practices for establishing and documenting this measurement property. METHODS: Topics addressed in the article include: definition and general description of content validity; PRO concept identification as the important first step in establishing content validity; instrument identification and the initial review process; key issues in qualitative methodology; and potential threats to content validity, with three case examples used to illustrate types of threats and how they might be resolved. A table of steps used to identify and evaluate an existing PRO instrument is provided, and figures are used to illustrate the meaning of content validity in relationship to instrument development and evaluation. RESULTS & RECOMMENDATIONS: Four important threats to content validity are identified: unclear conceptual match between the PRO instrument and the intended claim, lack of direct patient input into PRO item content from the target population in which the claim is desired, no evidence that the most relevant and important item content is contained in the instrument, and lack of documentation to support modifications to the PRO instrument. In some cases, careful review of the threats to content validity in a specific application may be reduced through additional well documented qualitative studies that specifically address the issue of concern. CONCLUSION: Published evidence of the content validity of a PRO instrument for an intended application is often limited. Such evidence is, however, important to evaluating the adequacy of a PRO instrument for the intended application. This article provides an overview of key issues involved in assessing and documenting content validity as it relates to using existing instruments in the drug approval process.

Can patients with asthma feel inhaler therapy working right away? Two clinical trials testing the effect of timing of assessment on patient perception.

BACKGROUND: Feeling a maintenance therapy work right away may provide positive reinforcement and may offer one way to improve adherence in patients with asthma. Precise measurement is required to accurately compare the presence of this effect across clinical trial treatment groups. METHODS: Two randomized, controlled studies tested whether timing of assessment (daily vs weekly, study 1; and predose vs postdose, study 2) influenced patients' reports of whether they can feel a medication working right away (perception), and their satisfaction with this perception (satisfaction). These 2-week US-based multicenter double-blind, parallel-group studies included patients > or = 18 years of age with mild to moderate persistent asthma. In each, patients were randomized to one of two drugs with different onset profiles: budesonide/formoterol pressurized metered-dose inhaler (pMDI) 80/4.5 microg x 2 inhalations (160/9 microg) twice daily or budesonide pMDI 80 microg x 2 inhalations (160 microg) twice daily. Patients were further randomized to complete previously validated perception and satisfaction questions in a cross-over fashion, either daily and weekly (N = 123) or predose and postdose (N = 134). Patient surveys also assessed perceptions of the onset of effect of medication and their value of these perceptions. RESULTS: No significant differences were observed in patients' reports of perception, either daily versus weekly or predose versus postdose. A statistically significant difference in satisfaction was found in study 1 only, favoring weekly recall (p < 0.05), with sensitivity analysis showing no difference by treatment group (p = 0.162). Across both studies, most patients (87%) who perceived their inhaler working right away (136 of 157 patients) identified positive airway sensations. Most patients reported that feeling their medication work right away is reassuring and would help them manage their asthma. CONCLUSION: Assessment timing has no effect on patient response to the perception of feeling a medication working right away. Differences found in satisfaction levels reported with weekly versus daily recall were consistent across treatment groups, indicating that no bias was introduced in favor of either treatment group. Patients characterized the perception of feeling a maintenance therapy working right away as easier breathing and reported this perception as beneficial to patient self-care.

Responsiveness of PROMIS® to change in chronic obstructive pulmonary disease.

BACKGROUND: Chronic obstructive pulmonary disease (COPD) is a progressive chronic disease characterized by airflow obstruction that leads to shortness of breath and substantial negative impacts on health-related quality of life (HRQL). The course of COPD includes periodic acute exacerbations that require changes in treatment and/or hospitalizations. This study was designed to examine the responsiveness of Patient-Reported Outcomes Measurement Information System® (PROMIS®) measures to changes associated with COPD exacerbation recovery. METHODS: A longitudinal analysis using mixed-effects models was conducted of people who were enrolled while stable (n = 100) and those who experienced an acute exacerbation (n = 85). PROMIS (physical function, pain interference, pain behavior, fatigue, anxiety, depression, anger, social roles, discretionary social activities, Global Health, dyspnea severity and dyspnea functional limitations) and COPD-targeted HRQL measures were completed at baseline and at 12 weeks. RESULTS: We administered PROMIS measures using computer adaptive testing (CAT), followed by administration of any remaining short form (SF) items that had not yet been administered by CAT. Examination of the difference between group differences from baseline to 12 weeks in the stable and exacerbation groups revealed that the exacerbation group changed (improved) significantly more than the stable group in anxiety (p < .001 to p < .01; f2 effect size [ES] = 0.023/0.021), fatigue (p < .0001; ES = 0.036/0.047) and social roles (p < .001 to p < .05; ES = 0.035/0.024). All effect sizes were small in magnitude and smaller than hypothesized. Depression was also statistically significant (p < .05, SF only) but the ES was trivial. For all other PROMIS domains, the differences were not significant and ES were trivial. CONCLUSIONS: This longitudinal study provides some support for the validity of the PROMIS fatigue, anxiety, and social roles domains in COPD, but further evaluation of responsiveness is warranted.

Perspectives on patient-reported outcomes : content validity and qualitative research in a changing clinical trial environment.

Patient-reported outcomes (PROs) represent the voice of the patient in drug and device evaluation. As such, the outcomes selected for evaluation must be relevant to the patient, and the instruments used to capture them must have sound measurement properties. This paper discusses the role of qualitative research methods in assuring PRO clarity and content validity in a clinical trial environment characterized by evolving regulatory policy, new advances in technology and increasingly diverse samples within global clinical trials. Three important PRO measurement issues influenced by these changes are addressed: (i) endpoint selection and instrument development; (ii) the adaptation of instruments for electronic administration; and (iii) conceptual equivalence of measures across diverse samples. These are viewed as interrelated issues of content validity that can be addressed through the appropriate and rigorous application of qualitative research methods. Focus groups, cognitive debriefing, user acceptance testing and translation methodologies are discussed as ways to address and document the content validity of PRO instruments and ensure the empirical data representing the voice of the patient is sound.

Evaluation of a respiratory symptom diary for clinical studies of idiopathic pulmonary fibrosis.

BACKGROUND: There are no validated patient diaries for evaluating respiratory symptoms in idiopathic pulmonary fibrosis (IPF). PURPOSE: To evaluate the performance properties of the chronic obstructive pulmonary disease (COPD) Evaluating Respiratory Symptoms™ (E-RS™: COPD) measure in patients with IPF. METHODS: Concept elicitation and cognitive interviews were conducted with IPF patients to evaluate content validity, including comprehensiveness, relevance, and interpretability of E-RS™ items in this patient population. Secondary analyses of IPF clinical study data were performed to evaluate the scoring structure of the tool. With modifications, reliability, validity, and responsiveness of the instrument (E-RS™: IPF) were evaluated. RESULTS: Qualitative interviews (n = 30) were conducted. During the elicitation interviews (n = 20), concept saturation for IPF respiratory symptoms was achieved; all respiratory symptoms covered by the E-RS™ were endorsed by ≥ 30% of the sample. During cognitive interviews (n = 10), all participants found the items interpretable and relevant. Factor analyses conducted via secondary analysis of IPF clinical study data identified no total score and four symptom scales: Chest, Breathlessness, Cough, and Sputum. Reliability of each scale was high (internal consistency [α] >0.85); 2-day reproducibility (ICC >0.88). Validity was supported through significant (P < 0.0001) relationships with the St. George's Respiratory Questionnaire (SGRQ), the University of California, San Diego Shortness of Breath Questionnaire (UCSD-SOBQ), and other variables. The scales were responsive to change when evaluated using SGRQ Symptoms, UCSD-SOBQ, and Patient Global Impression of Change as anchors (P < 0.01 to P < 0.0001). CONCLUSION: The E-RS™: IPF is a valid, reliable, and responsive tool for evaluating respiratory symptoms in patients with IPF.

The New York Heart Association Classes and functional status: what are we really measuring?

The New York Heart Association (NYHA) Classes are used to appraise the status of patients with heart disease and evaluate treatment outcomes in clinical and research settings. Ambiguity exists concerning the construct the Classes represent and the optimal way to capture and interpret the information. This article examines the NYHA Classes within the context of a published functional status framework by Leidy. The framework proposes that (1) physiological indicators provide information on capacity, (2) physical activity characterizes performance, and (3) symptoms accompanying activity offer insight into reserve. It is proposed that the NYHA Classes provide a summary statement of both the reduction in reserve accompanying a decline in capacity and the concomitant increase in capacity utilization required to maintain performance in patients with heart disease. This premise is illustrated quantitatively through secondary analysis of data from 22 patients with ischemic heart disease and left ventricular dysfunction.