Challenges in Research and Health Technology Assessment of Rare Disease Technologies: Report of the ISPOR Rare Disease Special Interest Group.

BACKGROUND: Successful development of new treatments for rare diseases (RDs) and their sustainable patient access require overcoming a series of challenges related to research and health technology assessment (HTA). These impediments, which may be unique to RDs or also apply to common diseases but are particularly pertinent in RDs, are diverse and interrelated. OBJECTIVE: To develop for the first time a catalog of primary impediments to RD research and HTA, and to describe the cause and effect of individual challenges. METHODS: Challenges were identified by an international 22-person expert working group and qualitative outreach to colleagues with relevant expertise. A broad range of stakeholder perspectives is represented. Draft results were presented at annual European and North American International Society for Pharmacoeconomics and Outcomes Research (ISPOR) congresses, and written comments were received by the 385-strong ISPOR Rare Disease Review Group from two rounds of review. Findings were refined and confirmed via targeted literature search. RESULTS: Research-related challenges linked to the low prevalence of RDs were categorized into those pertaining to disease recognition and diagnosis, evaluation of treatment effect, and patient recruitment for clinical research. HTA-related challenges were classified into issues relating to the lack of a tailored HTA method for RD treatments and uncertainty for HTA agencies and health care payers. CONCLUSIONS: Identifying and highlighting diverse, but interrelated, key challenges in RD research and HTA is an essential first step toward developing implementable and sustainable solutions. A collaborative multistakeholder effort is required to enable faster and less costly development of safe, efficacious, and appropriate new RD therapies that offer value for money.

Clinician-Reported Outcome Assessments of Treatment Benefit: Report of the ISPOR Clinical Outcome Assessment Emerging Good Practices Task Force.

A clinician-reported outcome (ClinRO) assessment is a type of clinical outcome assessment (COA). ClinRO assessments, like all COAs (patient-reported, observer-reported, or performance outcome assessments), are used to 1) measure patients' health status and 2) define end points that can be interpreted as treatment benefits of medical interventions on how patients feel, function, or survive in clinical trials. Like other COAs, ClinRO assessments can be influenced by human choices, judgment, or motivation. A ClinRO assessment is conducted and reported by a trained health care professional and requires specialized professional training to evaluate the patient's health status. This is the second of two reports by the ISPOR Clinical Outcomes Assessment-Emerging Good Practices for Outcomes Research Task Force. The first report provided an overview of COAs including definitions important for an understanding of COA measurement practices. This report focuses specifically on issues related to ClinRO assessments. In this report, we define three types of ClinRO assessments (readings, ratings, and clinician global assessments) and describe emerging good measurement practices in their development and evaluation. The good measurement practices include 1) defining the context of use; 2) identifying the concept of interest measured; 3) defining the intended treatment benefit on how patients feel, function, or survive reflected by the ClinRO assessment and evaluating the relationship between that intended treatment benefit and the concept of interest; 4) documenting content validity; 5) evaluating other measurement properties once content validity is established (including intra- and inter-rater reliability); 6) defining study objectives and end point(s) objectives, and defining study end points and placing study end points within the hierarchy of end points; 7) establishing interpretability in trial results; and 8) evaluating operational considerations for the implementation of ClinRO assessments used as end points in clinical trials. Applying good measurement practices to ClinRO assessment development and evaluation will lead to more efficient and accurate measurement of treatment effects. This is important beyond regulatory approval in that it provides evidence for the uptake of new interventions into clinical practice and provides justification to payers for reimbursement on the basis of the clearly demonstrated added value of the new intervention.

Rare Disease Terminology and Definitions-A Systematic Global Review: Report of the ISPOR Rare Disease Special Interest Group.

BACKGROUND: At present, there is no universal definition of rare disease. OBJECTIVE: To provide an overview of rare disease definitions currently used globally. METHODS: We systematically searched for definitions related to rare disease from organizations in 32 international jurisdictions. Descriptive statistics of definitions were generated and prevalence thresholds were calculated. RESULTS: We identified 296 definitions from 1109 organizations. The terms "rare disease(s)" and "orphan drug(s)" were used most frequently (38% and 27% of the definitions, respectively). Qualitative descriptors such as "life-threatening" were used infrequently. A prevalence threshold was specified in at least one definition in 88% of the jurisdictions. The average prevalence threshold across organizations within individual jurisdictions ranged from 5 to 76 cases/100,000 people. Most jurisdictions (66%) had an average prevalence threshold between 40 and 50 cases/100,000 people, with a global average of 40 cases/100,000 people. Prevalence thresholds used by different organizations within individual jurisdictions varied substantially. Across jurisdictions, umbrella patient organizations had the highest (most liberal) average prevalence threshold (47 cases/100,000 people), whereas private payers had the lowest threshold (18 cases/100,000 people). CONCLUSIONS: Despite variation in the terminology and prevalence thresholds used to define rare diseases among different jurisdictions and organizations, the terms "rare disease" and "orphan drug" are used most widely and the average prevalence threshold is between 40 and 50 cases/100,000 people. These findings highlight the existing diversity among definitions of rare diseases, but suggest that any attempts to harmonize rare disease definitions should focus on standardizing objective criteria such as prevalence thresholds and avoid qualitative descriptors.

Clinical Outcome Assessments: Conceptual Foundation-Report of the ISPOR Clinical Outcomes Assessment - Emerging Good Practices for Outcomes Research Task Force.

An outcome assessment, the patient assessment used in an endpoint, is the measuring instrument that provides a rating or score (categorical or continuous) that is intended to represent some aspect of the patient's health status. Outcome assessments are used to define efficacy endpoints when developing a therapy for a disease or condition. Most efficacy endpoints are based on specified clinical assessments of patients. When clinical assessments are used as clinical trial outcomes, they are called clinical outcome assessments (COAs). COAs include any assessment that may be influenced by human choices, judgment, or motivation. COAs must be well-defined and possess adequate measurement properties to demonstrate (directly or indirectly) the benefits of a treatment. In contrast, a biomarker assessment is one that is subject to little, if any, patient motivational or rater judgmental influence. This is the first of two reports by the ISPOR Clinical Outcomes Assessment - Emerging Good Practices for Outcomes Research Task Force. This report provides foundational definitions important for an understanding of COA measurement principles. The foundation provided in this report includes what it means to demonstrate a beneficial effect, how assessments of patients relate to the objective of showing a treatment's benefit, and how these assessments are used in clinical trial endpoints. In addition, this report describes intrinsic attributes of patient assessments and clinical trial factors that can affect the properties of the measurements. These factors should be considered when developing or refining assessments. These considerations will aid investigators designing trials in their choice of using an existing assessment or developing a new outcome assessment. Although the focus of this report is on the development of a new COA to define endpoints in a clinical trial, these principles may be applied more generally. A critical element in appraising or developing a COA is to describe the treatment's intended benefit as an effect on a clearly identified aspect of how a patient feels or functions. This aspect must have importance to the patient and be part of the patient's typical life. This meaningful health aspect can be measured directly or measured indirectly when it is impractical to evaluate it directly or when it is difficult to measure. For indirect measurement, a concept of interest (COI) can be identified. The COI must be related to how a patient feels or functions. Procedures are then developed to measure the COI. The relationship of these measurements with how a patient feels or functions in the intended setting and manner of use of the COA (the context of use) could then be defined. A COA has identifiable attributes or characteristics that affect the measurement properties of the COA when used in endpoints. One of these features is whether judgment can influence the measurement, and if so, whose judgment. This attribute defines four categories of COAs: patient reported outcomes, clinician reported outcomes, observer reported outcomes, and performance outcomes. A full description as well as explanation of other important COA features is included in this report. The information in this report should aid in the development, refinement, and standardization of COAs, and, ultimately, improve their measurement properties.

Validation of electronic systems to collect patient-reported outcome (PRO) data-recommendations for clinical trial teams: report of the ISPOR ePRO systems validation good research practices task force.

Outcomes research literature has many examples of high-quality, reliable patient-reported outcome (PRO) data entered directly by electronic means, ePRO, compared to data entered from original results on paper. Clinical trial managers are increasingly using ePRO data collection for PRO-based end points. Regulatory review dictates the rules to follow with ePRO data collection for medical label claims. A critical component for regulatory compliance is evidence of the validation of these electronic data collection systems. Validation of electronic systems is a process versus a focused activity that finishes at a single point in time. Eight steps need to be described and undertaken to qualify the validation of the data collection software in its target environment: requirements definition, design, coding, testing, tracing, user acceptance testing, installation and configuration, and decommissioning. These elements are consistent with recent regulatory guidance for systems validation. This report was written to explain how the validation process works for sponsors, trial teams, and other users of electronic data collection devices responsible for verifying the quality of the data entered into relational databases from such devices. It is a guide on the requirements and documentation needed from a data collection systems provider to demonstrate systems validation. It is a practical source of information for study teams to ensure that ePRO providers are using system validation and implementation processes that will ensure the systems and services: operate reliably when in practical use; produce accurate and complete data and data files; support management control and comply with any existing regulations. Furthermore, this short report will increase user understanding of the requirements for a technology review leading to more informed and balanced recommendations or decisions on electronic data collection methods.

Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 1--eliciting concepts for a new PRO instrument.

The importance of content validity in developing patient reported outcomes (PRO) instruments is stressed by both the US Food and Drug Administration and the European Medicines Agency. Content validity is the extent to which an instrument measures the important aspects of concepts that developers or users purport it to assess. A PRO instrument measures the concepts most significant and relevant to a patient's condition and its treatment. For PRO instruments, items and domains as reflected in the scores of an instrument should be important to the target population and comprehensive with respect to patient concerns. Documentation of target population input in item generation, as well as evaluation of patient understanding through cognitive interviewing, can provide the evidence for content validity. Developing content for, and assessing respondent understanding of, newly developed PRO instruments for medical product evaluation will be discussed in this two-part ISPOR PRO Good Research Practices Task Force Report. Topics include the methods for generating items, documenting item development, coding of qualitative data from item generation, cognitive interviewing, and tracking item development through the various stages of research and preparing this tracking for submission to regulatory agencies. Part 1 covers elicitation of key concepts using qualitative focus groups and/or interviews to inform content and structure of a new PRO instrument. Part 2 covers the instrument development process, the assessment of patient understanding of the draft instrument using cognitive interviews and steps for instrument revision. The two parts are meant to be read together. They are intended to offer suggestions for good practices in planning, executing, and documenting qualitative studies that are used to support the content validity of PRO instruments to be used in medical product evaluation.

Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: part 2--assessing respondent understanding.

The importance of content validity in developing patient reported outcomes (PRO) instruments is stressed by both the US Food and Drug Administration and the European Medicines Agency. Content validity is the extent to which an instrument measures the important aspects of concepts developers or users purport it to assess. A PRO instrument measures the concepts most relevant and important to a patient's condition and its treatment. For PRO instruments, items and domains as reflected in the scores of an instrument should be important to the target population and comprehensive with respect to patient concerns. Documentation of target population input in item generation, as well as evaluation of patient understanding through cognitive interviewing, can provide the evidence for content validity. Part 1 of this task force report covers elicitation of key concepts using qualitative focus groups and/or interviews to inform content and structure of a new PRO instrument. Building on qualitative interviews and focus groups used to elicit concepts, cognitive interviews help developers craft items that can be understood by respondents in the target population and can ultimately confirm that the final instrument is appropriate, comprehensive, and understandable in the target population. Part 2 details: 1) the methods for conducting cognitive interviews that address patient understanding of items, instructions, and response options; and 2) the methods for tracking item development through the various stages of research and preparing this tracking for submission to regulatory agencies. The task force report's two parts are meant to be read together. They are intended to offer suggestions for good practice in planning, executing, and documenting qualitative studies that are used to support the content validity of PRO instruments to be used in medical product evaluation.

Multinational trials-recommendations on the translations required, approaches to using the same language in different countries, and the approaches to support pooling the data: the ISPOR Patient-Reported Outcomes Translation and Linguistic Validation Good Research Practices Task Force report.

OBJECTIVES: With the internationalization of clinical trial programs, there is an increased need to translate and culturally adapt patient-reported outcome (PRO) measures. Although guidelines for good practices in translation and linguistic validation are available, the ISPOR Patient-Reported Outcomes Translation and Linguistic Validation Task Force identified a number of areas where they felt that further discussion around methods and best practices would be beneficial. The areas identified by the team were as follows: 1) the selection of the languages required for multinational trials; 2) the approaches suggested when the same language is required across two or more countries; and 3) the assessment of measurement equivalence to support the aggregation of data from different countries. METHODS: The task force addressed these three areas, reviewed the available literature, and had multiple discussions to develop this report. RESULTS: Decision aid tools have also been developed and presented for the selection of languages and the approaches suggested for the use of the same language in different countries. CONCLUSION: It is hoped that this report and the decision tools proposed will assist those involved with multinational trials to 1) decide on the translations required for each country; 2) choose the approach to use when the same language is spoken in more than one country; and 3) choose methods to gather evidence to support the pooling of data collected using different language versions of the same tool.