Best Practice Recommendations for Electronic Patient-Reported Outcome Dataset Structure and Standardization to Support Drug Development.

The electronic patient-reported outcome (ePRO) Dataset Structure and Standardization Project is a multistakeholder initiative formed by Critical Path Institute's PRO Consortium and Electronic Clinical Outcome Assessment (eCOA) Consortium to address issues related to ePRO dataset structure and standardization and to provide best practice recommendations for clinical trial sponsors and eCOA providers. Given the many benefits of utilizing electronic modes to capture PRO data, clinical trials are increasingly using these methods, yet there are challenges to using data generated by eCOA systems. Clinical Data Interchange Standards Consortium (CDISC) standards are used in clinical trials to ensure consistency in data collection, tabulation, and analysis and to facilitate regulatory submission. Currently, ePRO data are not required to follow a standard model, and the data models used often vary by eCOA provider and sponsor. This lack of consistency creates risks for programming and analysis and difficulties for analytics functions generating the required analysis and submission datasets. There is a disconnect between data standards used for study data submission and those used for data collection via case report forms and ePRO forms, which would be mitigated through the application of CDISC standards for ePRO data capture and transfer. The project was formed to collate and examine the issues arising from the lack of adoption of standardized approaches and this paper details recommendations to address those issues. Recommendations to address issues with ePRO dataset structure and standardization include adopting CDISC standards in the ePRO data platform, timely involvement of key stakeholders, ensuring ePRO controls are implemented, addressing issues of missing data early in development, ensuring quality control and validation of ePRO datasets, and use of read-only datasets.

Interpreting Within-Patient Changes on the EORTC QLQ-C30 and EORTC QLQ-LC13.

INTRODUCTION: When determining if changes on patient-reported outcome (PRO) scores in clinical trials convey a meaningful treatment benefit, statistical significance tests alone may not communicate the patient perspective. Appraising within-patient changes on PRO scores against established thresholds can determine if improvements or deteriorations experienced by individuals are meaningful. To evaluate the appropriateness of thresholds for interpreting meaningful improvements and deterioration within individuals on the European Organisation for Research and Treatment of Cancer (EORTC) 30-item core instrument (QLQ-C30) and 13-item lung cancer module (QLQ-LC13), a series of psychometric methods were applied to data from a phase III randomized controlled clinical trial in non-small cell lung cancer. METHODS: Anchor-based methods of empirical cumulative distribution functions and classification statistics were employed using change scores from Baseline to Week 7 using changes on the QLQ-C30 Global Health Status item as an anchor. Distribution-based methods of one-half standard deviation and standard error of measurement identified the minimum amount of change each domain score can reliably measure. RESULTS: While the correlations between the domain scores and the anchor item were modest in size (i.e., r ≥ 0.30 for only 5 of 24 domains), consideration of multiple methods along with the magnitude of possible step changes on the score allowed for patterns to emerge. The triangulation process planned a priori resulted in different methods being the source for different domain scores. Absolute values of the proposed thresholds ranged from 11.11 to 33.33, and all resulted in the same classifications for all EORTC domains, except QLQ-C30 Fatigue, as would the 10-point threshold that is traditionally used. CONCLUSION: This study confirms the appropriateness of the 10-point EORTC score threshold generally used by the field for interpreting within-patient changes, but the thresholds proposed from this study enhance interpretability by corresponding to only observable locations along the domain score scale.

Asthma Daytime Symptom Diary (ADSD) and Asthma Nighttime Symptom Diary (ANSD): Measurement Properties of Novel Patient-Reported Symptom Measures.

BACKGROUND: The Asthma Daytime Symptom Diary (ADSD) and the Asthma Nighttime Symptom Diary (ANSD) were developed to meet the need for standardized patient-reported measures of asthma symptoms to assess treatment trial outcomes in adults and adolescents. OBJECTIVE: To determine scoring and evaluate the measurement properties of the ADSD/ANSD. METHODS: Adolescents (12-17 years) and adults (18+ years) with asthma completed draft 8-item electronic versions of the ADSD/ANSD for 10 days alongside the Adult Asthma Symptom Daily Scales (AASDS) and a Patient Global Impression of Severity (PGIS). Using classical and modern psychometric methods, initial analyses evaluated the performance of ADSD/ANSD items to inform scoring. Subsequent analyses evaluated the reliability and validity of ADSD/ANSD scores. RESULTS: A demographically and clinically diverse sample (n = 130 adolescents; n = 89 adults) was recruited. Item performance was generally strong. However, items assessing chest pressure and mucus/phlegm demonstrated redundancy and poorer performance and were removed. Principal-components analysis, confirmatory factor analysis, and item response theory supported combining items to form 6-item total ADSD/ANSD scores. Internal consistency (α = 0.94-0.95) and test-retest reliability (intraclass correlation coefficient = 0.86-0.95) were strong. Strong correlations (r = 0.72-0.80) were observed between ADSD scores and AASDS items assessing asthma symptom frequency, bother, and impact on activities. Significant differences (P < .001) in mean ADSD/ANSD scores were observed between groups categorized by asthma severity (PGIS), asthma control, inhaler use, nebulizer use, activity limitations, and nighttime awakenings. CONCLUSIONS: The ADSD/ANSD items and scores demonstrated strong reliability and validity. Implementation of the measures in interventional studies will enable the evaluation of responsiveness and meaningful within-patient change.

Development of the Near Vision Presbyopia Task-based Questionnaire for use in evaluating the impact of presbyopia.

BACKGROUND: Presbyopia is a progressive condition that reduces the eye's ability to focus on near objects with increasing age. After a systematic literature review identified no existing presbyopia-specific patient-reported outcome (PRO) instruments meeting regulatory guidance, a new PRO instrument, the Near Vision Presbyopia Task-based Questionnaire (NVPTQ), was developed. RESULTS: To explore the patient experience with presbyopia, concept elicitation interviews were conducted with 20 presbyopic participants. The most frequently reported impacts were difficulty with reading menus/books/newspapers/magazines, reading on a cell phone/caller ID, and reading small print. Based on these results, a task-based PRO instrument (the NVPTQ) was developed instructing participants to complete four near-vision, paper-based reading tasks (book, newspaper, nutrition label, menu) under standardized settings, and subsequently assess their vision-related reading ability and associated satisfaction. The draft NVPTQ was cognitively debriefed with a sample of 20 presbyopes, which demonstrated that most participants interpreted the items as intended and endorsed the relevance of the concepts being assessed. After the qualitative research, the draft instrument was psychometrically tested using data from a Phase 2 study. Based on item-level analyses, all items in the NVPTQ demonstrated expected response option patterns and lacked substantial floor or ceiling effects. The reliability, validity, and responsiveness of the NVPTQ Performance and Satisfaction domain scores were assessed. All domains scores had large Cronbach's coefficient α values and good test-retest statistics, indicating that the scores are internally consistent and produce stable values over time. The pattern of correlations with a concurrent measure of visual functioning (National Eye Institute Visual Function Questionnaire 25) demonstrated that the NVPTQ domain scores were related to an alternative assessment of near-vision activities. The NVPTQ domain scores were able to distinguish between groups that were known to differ on the clinical outcome of uncorrected near visual acuity, supporting the construct validity of these scores. The NVPTQ domain scores showed evidence of responsiveness to change by being able to distinguish between groups defined as improved and not improved based on patient-reported and clinical outcomes. CONCLUSIONS: This research has resulted in a content-valid and psychometrically sound instrument designed to evaluate vision-related reading ability and satisfaction with vision-related reading ability. TRIAL REGISTRATION: ClinicalTrials.gov NCT02780115. Registered 23 May 2016, https://www.clinicaltrials.gov/ct2/show/NCT02780115?term=NCT02780115&draw=2&rank=1.

Development of the Presbyopia Impact and Coping Questionnaire.

INTRODUCTION: Presbyopia is a progressive, age-related visual condition that is characterized by reduced ability to focus on near/close objects, causing impacts on individuals' daily function and health-related quality of life. The Presbyopia Impact and Coping Questionnaire (PICQ) is a new patient-reported outcome (PRO) instrument that assesses presbyopia impact and use of coping behaviors among presbyopic individuals. METHODS: To document the impacts of presbyopia and associated coping behaviors, concept elicitation (CE) interviews were conducted with 20 presbyopic participants. Results from the CE interviews were used to develop draft items for additional testing. Following item generation, the draft PICQ was cognitively debriefed with 20 participants. Data from a phase 2 controlled clinical trial were used for psychometric analyses of the PICQ. The PICQ was administered at site visits throughout a 28-day treatment period. Confirmatory factor analysis (CFA) methods were used to guide the development of the scoring algorithm. The reliability (internal consistency, test-retest), construct validity (convergent and discriminant validity, known-groups methods), and responsiveness (Guyatt's responsiveness statistic [GRS]) of the PICQ scores were evaluated. Finally, anchor-based and distribution-based methods were used to inform thresholds for interpreting meaningful within-patient change. RESULTS: CE interviews identified the important and relevant presbyopia-related impacts and coping behaviors and 22 items were drafted and cognitively debriefed. Following minor revisions and item addition/deletion, a version of the PICQ including 23 items was subjected to psychometric testing. The analysis sample included 151 participants. The CFA established two PICQ domain scores, Coping and Impact, on 0-to-4 scales that demonstrate good model fit (root mean square error of approximation = 0.06, comparative fit index = 0.98, Tucker-Lewis index = 0.98, standardized root mean square = 0.07). Cronbach's alphas for the Coping and Impact scores were 0.89 and 0.84, respectively. Test-retest intraclass correlation coefficients were 0.77 for Coping and 0.67 for Impact. The pattern of results assessing construct validity was acceptable for the PICQ Coping and Impact scores, with the magnitude of correlations and effect sizes generally meeting a priori expectations. The corresponding GRS effect sizes for the PICQ Coping scores were -1.23 (i.e., large) for Patient Global Impression of Change (PGIC) and -0.72 (i.e., medium) for uncorrected near visual acuity (UNVA). The GRS effect sizes for the PICQ Impact scores were -0.60 (i.e., medium) for PGIC and -0.35 (i.e., small) for UNVA. Across three sets of anchor-based analyses for interpreting individual-level change, a responder threshold of -1.00 was identified for both PICQ Coping and PICQ Impact scores. CONCLUSIONS: The totality of evidence from the qualitative and quantitative research establishes that the PICQ scores produced are valid and reliable measures of presbyopia impacts and coping behaviors that are important and relevant for assessing presbyopia treatment outcomes. CLINICALTRIALS. GOV IDENTIFIER: NCT02780115; date of registration May 19, 2016.

Collection of Post-treatment PRO Data in Oncology Clinical Trials.

As patient-reported outcome (PRO) measures are being included more frequently in oncology clinical trials, regulatory and health technology assessment agencies have begun to request long-term, post-treatment PRO data to supplement traditional survival/progression endpoints. These data may be collected as part of cohort extension or registry studies to describe long-term outcomes of study participants after concluding their cancer treatment. While post-treatment PRO data may be expected to satisfy regulatory and payer expectations, significant practical barriers exist for the efficient incorporation of these data into oncology clinical trials, such as subject attrition, protocol deviations, and treatment crossover. The incorporation of post-treatment PRO assessments is a resource-intensive task requiring clear objectives for how the data will be analyzed and interpreted by both sponsors and regulators. Incorporating PRO data collection via electronic modalities (e.g., smartphone, web) may be a less expensive and more feasible option for incorporating long-term follow-up, reducing the frequency of manual study staff follow-up and expensive clinic visits. It is essential to include well-defined estimands for the statistical analysis, as well as to document limitations associated with the long-term follow-up data-collection approach. Analytical techniques will likely rely on descriptive and model-based statistics, and conclusions about treatment differences will likely be limited to preliminary findings of effectiveness (instead of efficacy). Finally, communications with health authorities and regulatory agencies regarding the LTFU study design and analysis should occur as early as possible to ensure that the PRO data to be collected offer an opportunity to properly evaluate the research question(s) of interest.

Psychometric Analysis of the Abdominal Score From the Diary for Irritable Bowel Syndrome Symptoms-Constipation Using Phase IIb Clinical Trial Data.

OBJECTIVES: The Diary for Irritable Bowel Syndrome Symptoms-Constipation (DIBSS-C) has been developed to assess the core signs and symptoms of irritable bowel syndrome with constipation (IBS-C). This article presents the psychometric evaluation of the DIBSS-C abdominal score. METHODS: Data for these analyses are from a multicenter phase IIb study in IBS-C patients (NCT02559206). Subjects completed a number of assessments via handheld electronic diary throughout the study. The analyses used the intent-to-treat population and were blinded to randomized treatment group. The analyses evaluated the reliability, validity, and responsiveness of the DIBSS-C abdominal score; identified an appropriate scoring algorithm; and determined thresholds for interpreting clinically meaningful changes at the individual level. RESULTS: The correlations between the DIBSS-C abdominal symptom items (ie, abdominal pain, discomfort, and bloating) were strong (>0.75). Cronbach's alpha for the abdominal symptom severity items was very strong (.94), indicating that the 3 abdominal symptom items produce a reliable score. The intraclass correlation coefficient for the abdominal score was 0.82, exceeding the threshold of 0.70 and indicating good test-retest reliability. Guyatt's responsiveness statistic values all exceeded the threshold for a large effect of 0.80, so the DIBSS-C abdominal score can be considered highly responsive to change. Triangulation across 3 sets of anchor-based analyses indicated that a threshold of -2.0 points on the abdominal score is an appropriate threshold for identifying meaningful change. CONCLUSIONS: Overall, this study provides evidence that the DIBSS-C abdominal score is valid, reliable, responsive to change, and interpretable for assessing treatment benefit in patients with IBS-C.

Psychometric evaluation of the PROMIS® Depression Item Bank: an illustration of classical test theory methods.

BACKGROUND: Psychometric theory offers a range of tests that can be used as supportive evidence of both validity and reliability of instruments aimed at measuring patient-reported outcomes (PRO). The aim of this paper is to illustrate psychometric tests within the Classical Test Theory (CTT) framework, comprising indices that are frequently applied to assess item- and scale-level psychometric properties of PRO instruments. METHODS: Using data on the PROMIS Depression Item Bank, typical CTT indices for the assessment of psychometric properties are illustrated, including content validity, item-level data exploration, reliability, and construct validity, particularly confirmatory factor analysis, to test the unidimensionality assumption underlying the item bank. Analyses are carried out on an original item set of 51 depression items, the final (official) PROMIS Depression Item Bank consisting of 28 items, and an 8-item short form. RESULTS: The analyses reported provide an informative illustration on how item- and scale-level reliability and validity statistics can be used to assess the psychometric quality of a PRO instrument. The results illustrate how the reported statistics can be used for item selection from an item pool (here: 51 items). Both the (final) 28-item bank and the 8-item short form show good psychometric properties supporting the high quality of individual items and the unidimensionality assumption of the item bank. CONCLUSIONS: It is our hope that our illustration of CTT methods, in conjunction with two companion papers illustrating modern test theory methods, will help researchers to confidently apply a range of statistical tests to evaluate item- and scale-level psychometric performance of PRO instruments.

Moving from significance to real-world meaning: methods for interpreting change in clinical outcome assessment scores.

PURPOSE: Clinical outcome assessments (COAs) require evidence not only of reliability, validity, and ability to detect change, but also a definition of what constitutes a meaningful change on the instrument. The responder definition specifies the amount of change on the COA that may be interpreted as a treatment benefit and is critical for interpreting what constitutes a meaningful change on the COA scores. However, the literature that describes methods for developing and applying responder definitions can be difficult to navigate. Clear and concise guidelines regarding which methods to apply under what circumstances and how to interpret the results are lacking. This article provides a guide to the variety of available methods and issues that should be considered when establishing responder definitions for interpreting meaningful changes in COA scores. METHODS: An overview is provided for selecting anchors, developing study designs, planning psychometric analyses, using psychometric results to set responder thresholds, and applying responder thresholds in demonstrating treatment efficacy. RESULTS: There are a variety of anchor-based methods for consideration, but they all rely on a preference for strongly related and easily interpretable anchors. The benefits of applying multiple anchors and multiple analytic methods are discussed. The process of triangulation can synthesize results across multiple sources to gain confidence in a proposed responder definition. Though a link to meaningfulness from the patient's perspective is absent, distribution-based methods provide lower bound estimates of score precision and have a role in triangulation. Responder definitions are typically required within regulatory review, but their application may differ across clinical trial programs. CONCLUSIONS: By careful planning of anchor selection, study design, and psychometric methods, COA researchers can establish defensible responder thresholds that ultimately aid patients and clinicians in making informed treatment decisions.

Idio Scale Judgment: evaluation of a new method for estimating responder thresholds.

PURPOSE: To pilot the newly developed Idio Scale Judgment method for estimating the amount of score change that matters to patients (i.e., change thresholds). METHODS: An online panel of 500 participants diagnosed with multiple sclerosis (MS) responded to the Neuro-QoL fatigue scale and to demographic and clinical questions. Participants compared their own fatigue to that represented by seven short form summaries (SFSs) that were located relatively close to their own fatigue levels. They judged these as representing the same, greater, or less fatigue than their own. We calculated the distances between patients' own levels of fatigue and the location of SFSs they endorsed as a change that would make a difference in daily life. These distances were used as estimates of change thresholds. Logically inconsistent judgments were tabulated and associations with clinical and demographic variables were estimated. RESULTS: Change thresholds based on mean individual thresholds for consequential change were calculated for improvement (-3.5) and worsening (3.2). The majority of participants had no logically inconsistent judgments (69%). Having one or more reversals in judgment was not significantly associated with education or fatigue score, but was weakly associated with age, gender, and MS type and moderately associated with ratings of confidence in SFS comparisons. CONCLUSIONS: As piloted, Idio Scale Judgment had a number of design strengths. Participants made comparisons to levels of fatigue that were within range of their own, and their judgments were contextualized in personally relevant consequences. The design lends itself to collection of data in large samples allowing evaluation of the range of judgments. Some study limitations could be mitigated with modifications. We concluded that the Idio Scale Judgment has substantial promise as a new tool for estimating change thresholds.

Impact of Pediatric Acute Otitis Media on Child and Parental Quality of Life and Associated Productivity Loss in Malaysia: A Prospective Observational Study.

BACKGROUND: Acute otitis media (AOM) affects both child and parental quality of life (QoL). Data on QoL associated with AOM in Malaysia is sparse, and the burden of indirect costs have not been previously reported. OBJECTIVE: To determine the effect of pediatric AOM on child and parental QoL in Malaysia and its economic impact (indirect costs). METHODS: We utilized a set of QoL questionnaires (PAR-AOM-QOL, OM-6, and EQ-5D) combined with questions addressing work/productivity loss and financial costs associated with caring for a child during his or her illness in an observational, multicenter, prospective study. RESULTS: One hundred and ten AOM patients aged ≤5 years were included in the analysis. The majority of respondents were the patient's mother. Parental QoL was negatively affected for both emotional and daily disturbance scales, but the level of disturbance was low. Using OM-6, the greatest negative impact was on the child's QoL, followed by caregiver concerns, physical suffering, and emotional distress. Using EQ-5D, a moderately positive relationship between parents' emotional disturbance and daily disturbance, and a weak, negative correlation between parental emotional disturbance and parental health status was found. Parents with paid employment took an average of 21 h from work to care for their child, at an average cost of 321.8 Malaysian ringgit (US$97) in addition to their contribution to direct medical costs. Productivity losses whilst at work, uncompensated wage losses, and leisure time losses are also reported. CONCLUSIONS: This study found that AOM is associated with some negative impact on parental QoL and significant economic impact at both patient and societal levels. The findings provide useful data on healthcare resource utilization and disease burden of AOM in Malaysia.

Development of the multiple sclerosis (MS) early mobility impairment questionnaire (EMIQ).

The Early Mobility Impairment Questionnaire (EMIQ) was developed to facilitate early identification of mobility impairments in multiple sclerosis (MS) patients. We describe the initial development of the EMIQ with a focus on the psychometric evaluation of the questionnaire using classical and item response theory methods. The initial 20-item EMIQ was constructed by clinical specialists and qualitatively tested among people with MS and physicians via cognitive interviews. Data from an observational study was used to make additional updates to the instrument based on exploratory factor analysis (EFA) and item response theory (IRT) analysis, and psychometric analyses were performed to evaluate the reliability and validity of the final instrument's scores and screening properties (i.e., sensitivity and specificity). Based on qualitative interview analyses, a revised 15-item EMIQ was included in the observational study. EFA, IRT and item-to-item correlation analyses revealed redundant items which were removed leading to the final nine-item EMIQ. The nine-item EMIQ performed well with respect to: test-retest reliability (ICC = 0.858); internal consistency (α = 0.893); convergent validity; and known-groups methods for construct validity. A cut-point of 41 on the 0-to-100 scale resulted in sufficient sensitivity and specificity statistics for viably identifying patients with mobility impairment. The EMIQ is a content valid and psychometrically sound instrument for capturing MS patients' experience with mobility impairments in a clinical practice setting. Additional research is suggested to further confirm the EMIQ's screening properties over time.

Interpreting Change in Scores on Patient-Reported Outcome Instruments.

Interpreting change in scores on patient-reported outcome instruments is a key aspect of instrument development. Without interpretation guidelines, the clinical meaning of significant improvements observed within a treatment group cannot be ascertained. While the field has contemplated this topic for several decades, there remains inconsistency in terminology, methods, and application. Careful selection of methods can result in determining when change is meaningful, but researchers must keep an open mind to the methods that best fit their study and instrument. In many cases, anchor-based methods are appropriate, but the statistical model that evaluates them should be defensible (eg, linear regression, repeated-measures modeling, logistic regression). Sometimes, researchers must entertain the use of novel methods that may be more appropriate for their planned studies and instrument (eg, standard setting, exit interviews, conjoint analysis). The selection of the method is best supported by clear, transparent communication with the regulatory agency to ensure that the method can support its goals.

Patient-reported outcomes for US oncology labeling: review and discussion of score interpretation and analysis methods.

This paper describes ways to approach the conceptual and practical challenges associated with interpreting the clinical meaning of scores produced by patient reported outcome (PRO) questionnaires, particularly when used to inform efficacy decisions for regulatory approval for oncology products. Score interpretation estimates are not inherent to PRO questionnaires per se, instead, vary dependent upon sample and study design characteristics. Scores from PRO measures can be interpreted at the individual and group level, and each carries its own set of statistics for evaluating differences. Oncology researchers have a variety of methods and data analytic strategies available to support their score interpretation needs, which should be considered in the context of their a priori knowledge of the target patient population, the hypothesized effects of treatment, the study design and assessment schedule, and the inferences and decisions to be made from the PRO data.

Three-year patient-reported visual function outcomes in diabetic macular edema managed with ranibizumab: the RESTORE extension study.

OBJECTIVE: To determine the impact of ranibizumab 0.5 mg on patient-reported visual function over 36 months in individuals with visual impairment from diabetic macular edema. METHODS: RESTORE comprises a phase 3, randomized, multicenter, 12 month core study and a 24 month open-label extension study. Eyes assigned to ranibizumab in the core study received ranibizumab for 36 months; eyes assigned to laser monotherapy in the core study received ranibizumab during the extension. The primary outcome was least-squares mean change in National Eye Institute 25-item Visual Functioning Questionnaire (NEI VFQ-25) overall composite and subscale scores. RESULTS: Of 303 core study participants, 240 (79%) entered the extension, comprising 83 (35%) participants initially assigned to ranibizumab, 83 (35%) assigned to ranibizumab plus laser combination therapy, and 74 (31%) assigned to laser monotherapy. Least-squares mean (standard error) change in NEI VFQ-25 composite score from baseline to month 12 (+5.9 [1.5]; +5.0 [1.5], for the ranibizumab and combination therapy groups, respectively) decreased by month 36 (+4.1 [1.7]; +4.0 [1.7], respectively, from baseline to month 36) following reduced injection frequency relative to the core study. At 36 months, the least-squares mean (standard error) change in the laser monotherapy group was similar to that in the ranibizumab groups (+4.1 [1.8]). Most subscale scores showed outcomes similar to that for the composite score. The greatest NEI VFQ-25 gains were consistently observed in participants for whom the study eye was the better-seeing eye. LIMITATIONS: Patients entering the extension were not randomized, and 21% of the core study participants did not enter the extension, which may have affected the results. CONCLUSIONS: Gains in patient-reported visual function at month 12 among eyes receiving ranibizumab in the core study decreased slightly by 36 months. Eyes originally receiving laser monotherapy for 12 months then ranibizumab for 24 months achieved similar gains by 36 months to eyes receiving ranibizumab for 36 months. TRIAL REGISTRATION: ClinicalTrials.gov: NCT00687804 and NCT00906464.

Using classical test theory, item response theory, and Rasch measurement theory to evaluate patient-reported outcome measures: a comparison of worked examples.

OBJECTIVE: To provide comparisons and a worked example of item- and scale-level evaluations based on three psychometric methods used in patient-reported outcome development-classical test theory (CTT), item response theory (IRT), and Rasch measurement theory (RMT)-in an analysis of the National Eye Institute Visual Functioning Questionnaire (VFQ-25). METHODS: Baseline VFQ-25 data from 240 participants with diabetic macular edema from a randomized, double-masked, multicenter clinical trial were used to evaluate the VFQ at the total score level. CTT, RMT, and IRT evaluations were conducted, and results were assessed in a head-to-head comparison. RESULTS: Results were similar across the three methods, with IRT and RMT providing more detailed diagnostic information on how to improve the scale. CTT led to the identification of two problematic items that threaten the validity of the overall scale score, sets of redundant items, and skewed response categories. IRT and RMT additionally identified poor fit for one item, many locally dependent items, poor targeting, and disordering of over half the response categories. CONCLUSIONS: Selection of a psychometric approach depends on many factors. Researchers should justify their evaluation method and consider the intended audience. If the instrument is being developed for descriptive purposes and on a restricted budget, a cursory examination of the CTT-based psychometric properties may be all that is possible. In a high-stakes situation, such as the development of a patient-reported outcome instrument for consideration in pharmaceutical labeling, however, a thorough psychometric evaluation including IRT or RMT should be considered, with final item-level decisions made on the basis of both quantitative and qualitative results.

Unique Challenges in Development, Psychometric Evaluation, and Interpretation of Daily and Event Diaries as Endpoints in Clinical Trials.

By bringing data collection closer to real time and minimizing recall bias, patient diaries or event-driven logs offer substantial benefits over retrospective questionnaires for many patient-reported disease concepts. Such assessments are increasingly used to support primary and secondary endpoints in randomized controlled trials. These diaries have the potential to provide more reliable and valid assessment of patients' subjective experiences of symptoms and disease events. However, use of such diaries presents significant and unique challenges depending on the context of use. Of note, while symptom-related label claims are those most frequently granted by regulatory authorities, no guidance specific to support the development, psychometric evaluation, and interpretation of endpoints derived from patient diaries exists. This article provides an overview of key methodological, statistical, and clinical considerations for implementation of patient diaries with a regulatory perspective in mind. Approaches and solutions covered in this article include (1) techniques to establish content validity based on obtaining qualitative insights in naturalistic settings and real-life experience of diary completion, (2) demonstration of psychometric properties with respect to day-to-day variability, and (3) aggregation of data from multiple days/events to move from items to endpoints. The importance of the patients' engagement is highlighted in order to help overcome these challenges throughout all stages of diary and endpoint development and evaluation. This article can inform researchers who are developing or implementing patient diaries as clinical trial endpoints to ensure that the nuances of this mode of data collection are considered in the development of endpoints and prior to regulatory interactions.

Psychometric evaluation of the Sheehan Disability Scale in adult patients with attention-deficit/hyperactivity disorder.

Inattention and impulsivity symptoms are common among adults with attention-deficit/hyperactivity disorder (ADHD), which can lead to difficulty concentrating, restlessness, difficulty completing tasks, disorganization, impatience, and impulsiveness. Many adults with ADHD find it difficult to focus and prioritize. Resulting outcomes, such as missed deadlines and forgotten engagements, may ultimately impact the ability to function at work, school, home, or in a social environment. The European Medicines Agency guidelines for evaluating medicinal products for ADHD recommend inclusion of both functional outcomes, such as school, social, or work functioning, and outcomes related to symptoms of ADHD in clinical studies of novel medication primary efficacy endpoints. Due to its performance in other disease areas and the relevance of its items as evidenced by content validity analyses, the Sheehan Disability Scale (SDS) was chosen to assess functional impairment in ADHD. The aim of this study was to investigate the psychometric properties of the SDS, used as a brief measure of functional impairment in a number of psychiatric disorders, in adult patients with ADHD. To the authors' knowledge, this is the first study to evaluate the reliability of the SDS (based on Cronbach's coefficient alpha and test-retest reliability), its validity (construct and known-groups validity), and its ability to detect change in this patient population. This study also established a preliminary responder definition for the SDS in this study population to determine when change can be considered clinically beneficial in a clinical trial setting. The psychometric results support the use of the SDS subscales (items 1-3) and total score (sum of items 1-3) in an ADHD population. In addition, the evaluation provides evidence for a three-point preliminary responder definition for the SDS and further evidence of its responsiveness in adults with ADHD. Altogether, the results indicate that the SDS is a simple and easy-to-score scale that would have great utility in future clinical trials for monitoring functional impairment in adults with ADHD.