Outcome pre-specification requires sufficient detail to guard against outcome switching in clinical trials: a case study.

BACKGROUND: Pre-specification of outcomes is an important tool to guard against outcome switching in clinical trials. However, if the outcome is not sufficiently clearly defined, then different definitions could be applied and analysed, with only the most favourable result reported. METHODS: In order to assess the impact that differing outcome definitions could have on treatment effect estimates, we re-analysed data from TRIGGER, a cluster randomised trial comparing two red blood cell transfusion strategies for patients with acute upper gastrointestinal bleeding. We varied several aspects of the definition of further bleeding: (1) the criteria for what constitutes a further bleeding episode; (2) how further bleeding is assessed; and (3) the time-point at which further bleeding is measured. RESULTS: There were marked discrepancies in the estimated odds ratios (OR) (range 0.23-0.94) and corresponding P values (range < 0.001-0.89) between different outcome definitions. At the extremes, differing outcome definitions led to markedly different conclusions; one definition led to very little evidence of a treatment effect (OR = 0.94, 95% confidence interval [CI] = 0.37-2.40, P = 0.89), while another led to very strong evidence of a treatment effect (OR = 0.23, 95% CI = 0.11-0.50, P < 0.001). CONCLUSIONS: Outcomes should be pre-specified in sufficient detail to avoid differing definitions being analysed and only the most favourable result being reported. TRIAL REGISTRATION: Clinical Trials.gov, NCT02105532 . Registered on 7 April 2014.

Sample size implications of mortality definitions in sepsis: a retrospective cohort study.

BACKGROUND: Many randomized controlled trials (RCTs) employ mortality at a given time as a primary outcome. There are at least three common ways to measure 90-day mortality: first, all-location mortality, that is, all-cause mortality within 90 days of randomization at any location. Second, ARDSnet mortality is death in a healthcare facility of greater intensity than the patient was in prior to the hospitalization during which they were randomized. Finally, in-hospital mortality is death prior to discharge from the primary hospitalization of randomization. Data comparing the impact of these different measurements on sample size are lacking. We evaluated the extent to which event rates vary by mortality definition. METHODS: This was a retrospective cohort study of 30,691 patients hospitalized at Veterans Affairs (VA) hospitals for sepsis during 2009. 12,727 (41.5%) received care in an ICU setting. For each patient, we measured event rates for three different 90-day mortality outcomes: all-location mortality, ARDSnet mortality, and in-hospital mortality. We also calculated sample sizes necessary to power an example RCT given those event rates. RESULTS: At 90 days, all-location mortality was 26.4% (95% CI 25.9-26.9%), ARDSnet mortality was 19.2% (95% CI 18.8-19.7%), and in-hospital mortality was 13.4% (95% CI 13.0-13.8%) (p < 0.01 all comparisons). These respective event rates result in different required sample sizes to achieve a 20% relative reduction in mortality with 80% power and a 5% false positive rate. Such a trial of VA sepsis patients would require 2080 patients for all-location mortality, 3080 for ARDSnet mortality, and 4796 for in-hospital mortality. Among sepsis patients mechanically ventilated in an ICU, 2438 experienced all-location mortality (46.2% [95% CI 44.8-47.5%]), 2181 experienced ARDSnet mortality (41.3% [95% CI 40.0-42.6%]), and 1894 experienced in-hospital mortality (36.0% [95% CI 34.7-37.3%]). CONCLUSIONS: Event rates vary substantially in sepsis patients based on the chosen 90-day mortality definition. This could have important implications for RCT design trade-offs.

Quality of reporting of outcomes in phase III studies of pulmonary tuberculosis: a systematic review.

BACKGROUND: Despite more than 60 years of clinical trials, tuberculosis (TB) still causes a high global burden of mortality and morbidity. Treatment currently requires multiple drugs in combination, taken over a prolonged period. New drugs are needed to shorten treatment duration, prevent resistance and reduce adverse events. However, to improve on current methodology in drug development, a more complete understanding of the existing clinical evidence base is required. METHODS: A systematic review was undertaken to summarise outcomes reported in phase III trials of patients with newly diagnosed pulmonary TB. A systematic search of databases (PubMed, MEDLINE, EMBASE, CENTRAL and LILACs) was conducted on 30 November 2017 to retrieve relevant peer-reviewed articles. Reference lists of included studies were also searched. This systematic review considered all reported outcomes. RESULTS: Of 248 included studies, 229 considered "on-treatment" outcomes whilst 148 reported "off-treatment" outcomes. There was wide variation and ambiguity in the definition of reported outcomes, including their relationship to treatment and in the time points evaluated. Additional challenges were observed regarding the analysis approach taken (per protocol versus intention to treat) and the varying durations of "intensive" and "continuation" phases of treatment. Bacteriological outcomes were most frequently reported but radiological and clinical data were often included as an implicit or explicit component of the overall definition of outcome. CONCLUSIONS: Terminology used to define long-term outcomes in phase III trials is inconsistent, reflecting evolving differences in protocols and practices. For successful future cumulative meta-analysis, the findings of this review suggest that greater availability of individual patient data and the development of a core outcome set would be desirable. In the meantime, we propose a simple and logical approach which should facilitate combination of key evidence and inform improvements in the methodology of TB drug development and clinical trials.

The search for an elusive cutoff remains: Problems of binary classification of heavy drinking as an endpoint for alcohol clinical trials.

BACKGROUND: To examine whether a clinically meaningful alcohol consumption cutoff can be created for clinical samples, we used receiver operator characteristic (ROC) curves to derive gender-specific consumption cutoffs that maximized sensitivity and specificity in the prediction of a wide range of negative consequences from drinking. METHODS: We conducted secondary data analyses using data from two large clinical trials targeting alcohol use disorders: Project MATCH (n=1726) and COMBINE (n=1383). RESULTS: In both studies, we found that the ideal cutoff for men and women that maximized sensitivity/specificity varied substantially both across different alcohol consumption variables and alcohol consequence outcomes. Further, the levels of sensitivity/specificity were poor across all consequences. CONCLUSIONS: These results fail to provide support for a clinically meaningful alcohol consumption cutoff and suggest that binary classification of levels of alcohol consumption is a poor proxy for maximizing sensitivity/specificity in the prediction of negative consequences from drinking. Future research examining consumption-consequence associations should take advantage of continuous measures of alcohol consumption and alternative approaches for assessing the link between levels of consumption and consequences (e.g., ecological momentary assessment). Clinical researchers should consider focusing more directly on the consequences they aim to reduce instead of relying on consumption as a proxy for more clinically meaningful outcomes.

Endpoint design for future renal denervation trials - Novel implications for a new definition of treatment response to renal denervation.

BACKGROUND: Defining an adequate endpoint for renal denervation trials represents a major challenge. A high inter-individual and intra-individual variability of blood pressure levels as well as a partial or total non-adherence on antihypertensive drugs hamper treatment evaluations after renal denervation. Blood pressure measurements at a single point in time as used as primary endpoint in most clinical trials on renal denervation, might not be sufficient to discriminate between patients who do or do not respond to renal denervation. METHODS: We compared the traditional responder classification (defined as systolic 24-hour blood pressure reduction of -5mmHg six months after renal denervation) with a novel definition of an ideal respondership (based on a 24h blood pressure reduction at no point in time, one, or all follow-up timepoints). RESULTS: We were able to re-classify almost a quarter of patients. Blood pressure variability was substantial in patients traditionally defined as responders. On the other hand, our novel classification of an ideal respondership seems to be clinically superior in discriminating sustained from pseudo-response to renal denervation. CONCLUSION: Based on our observations, we recommend that the traditional response classification should be reconsidered and possibly strengthened by using a composite endpoint of 24h-BP reductions at different follow-up-visits.

Clinical Outcome Assessments: Conceptual Foundation-Report of the ISPOR Clinical Outcomes Assessment - Emerging Good Practices for Outcomes Research Task Force.

An outcome assessment, the patient assessment used in an endpoint, is the measuring instrument that provides a rating or score (categorical or continuous) that is intended to represent some aspect of the patient's health status. Outcome assessments are used to define efficacy endpoints when developing a therapy for a disease or condition. Most efficacy endpoints are based on specified clinical assessments of patients. When clinical assessments are used as clinical trial outcomes, they are called clinical outcome assessments (COAs). COAs include any assessment that may be influenced by human choices, judgment, or motivation. COAs must be well-defined and possess adequate measurement properties to demonstrate (directly or indirectly) the benefits of a treatment. In contrast, a biomarker assessment is one that is subject to little, if any, patient motivational or rater judgmental influence. This is the first of two reports by the ISPOR Clinical Outcomes Assessment - Emerging Good Practices for Outcomes Research Task Force. This report provides foundational definitions important for an understanding of COA measurement principles. The foundation provided in this report includes what it means to demonstrate a beneficial effect, how assessments of patients relate to the objective of showing a treatment's benefit, and how these assessments are used in clinical trial endpoints. In addition, this report describes intrinsic attributes of patient assessments and clinical trial factors that can affect the properties of the measurements. These factors should be considered when developing or refining assessments. These considerations will aid investigators designing trials in their choice of using an existing assessment or developing a new outcome assessment. Although the focus of this report is on the development of a new COA to define endpoints in a clinical trial, these principles may be applied more generally. A critical element in appraising or developing a COA is to describe the treatment's intended benefit as an effect on a clearly identified aspect of how a patient feels or functions. This aspect must have importance to the patient and be part of the patient's typical life. This meaningful health aspect can be measured directly or measured indirectly when it is impractical to evaluate it directly or when it is difficult to measure. For indirect measurement, a concept of interest (COI) can be identified. The COI must be related to how a patient feels or functions. Procedures are then developed to measure the COI. The relationship of these measurements with how a patient feels or functions in the intended setting and manner of use of the COA (the context of use) could then be defined. A COA has identifiable attributes or characteristics that affect the measurement properties of the COA when used in endpoints. One of these features is whether judgment can influence the measurement, and if so, whose judgment. This attribute defines four categories of COAs: patient reported outcomes, clinician reported outcomes, observer reported outcomes, and performance outcomes. A full description as well as explanation of other important COA features is included in this report. The information in this report should aid in the development, refinement, and standardization of COAs, and, ultimately, improve their measurement properties.

Guidelines for time-to-event end point definitions in breast cancer trials: results of the DATECAN initiative (Definition for the Assessment of Time-to-event Endpoints in CANcer trials)†.

BACKGROUND: Using surrogate end points for overall survival, such as disease-free survival, is increasingly common in randomized controlled trials. However, the definitions of several of these time-to-event (TTE) end points are imprecisely which limits interpretation and cross-trial comparisons. The estimation of treatment effects may be directly affected by the definitions of end points. The DATECAN initiative (Definition for the Assessment of Time-to-event Endpoints in CANcer trials) aims to provide recommendations for definitions of TTE end points. We report guidelines for randomized cancer clinical trials (RCTs) in breast cancer. PATIENTS AND METHODS: A literature review was carried out to identify TTE end points (primary or secondary) reported in publications of randomized trials or guidelines. An international multidisciplinary panel of experts proposed recommendations for the definitions of these end points based on a validated consensus method that formalize the degree of agreement among experts. RESULTS: Recommended guidelines for the definitions of TTE end points commonly used in RCTs for breast cancer are provided for non-metastatic and metastatic settings. CONCLUSION: The use of standardized definitions should facilitate comparisons of trial results and improve the quality of trial design and reporting. These guidelines could be of particular interest to those involved in the design, conducting, reporting, or assessment of RCT.

Guidelines for time-to-event end point definitions in sarcomas and gastrointestinal stromal tumors (GIST) trials: results of the DATECAN initiative (Definition for the Assessment of Time-to-event Endpoints in CANcer trials)†.

BACKGROUND: The use of potential surrogate end points for overall survival, such as disease-free survival (DFS) or time-to-treatment failure (TTF) is increasingly common in randomized controlled trials (RCTs) in cancer. However, the definition of time-to-event (TTE) end points is rarely precise and lacks uniformity across trials. End point definition can impact trial results by affecting estimation of treatment effect and statistical power. The DATECAN initiative (Definition for the Assessment of Time-to-event End points in CANcer trials) aims to provide recommendations for definitions of TTE end points. We report guidelines for RCT in sarcomas and gastrointestinal stromal tumors (GIST). METHODS: We first carried out a literature review to identify TTE end points (primary or secondary) reported in publications of RCT. An international multidisciplinary panel of experts proposed recommendations for the definitions of these end points. Recommendations were developed through a validated consensus method formalizing the degree of agreement among experts. RESULTS: Recommended guidelines for the definition of TTE end points commonly used in RCT for sarcomas and GIST are provided for adjuvant and metastatic settings, including DFS, TTF, time to progression and others. CONCLUSION: Use of standardized definitions should facilitate comparison of trials' results, and improve the quality of trial design and reporting. These guidelines could be of particular interest to research scientists involved in the design, conduct, reporting or assessment of RCT such as investigators, statisticians, reviewers, editors or regulatory authorities.