Assessing and communicating heterogeneity of treatment effects for patient subpopulations: Panel discussion on considerations in design and analysis.

Clinical trials are primarily conducted to understand the average effects treatments have on patients. However, patients are heterogeneous in the severity of the condition and in ways that affect what treatment effect they can expect. It is therefore important to understand and characterize how treatment effects vary. The design and analysis of clinical studies play critical roles in evaluating and characterizing heterogeneous treatment effects. This panel discussed considerations in design and analysis under the recognition that there are heterogeneous treatment effects across subgroups of patients. Panel members discussed many questions including: What is a good estimate of the treatment effect in me, a 65-year-old, bald, Caucasian-American, male patient? What magnitude of heterogeneity of treatment effects (HTE) is sufficiently large to merit attention? What role can prior evidence about HTE play in confirmatory trial design and analysis? Is there anything described in the 21st Century Cures Act that would benefit from greater attention to HTE? An example of a Bayesian approach addressing multiplicity when testing for treatment effects in subgroups will be provided. We can do more or better at understanding heterogeneous treatment effects and providing the best information on heterogeneous treatment effects.

Defining treatment effects: A regulatory perspective.

The proposed addendum to the International Conference on Harmonization document, Statistical Principles for Clinical Trials, can be read in two ways. There is a new framework for talking about estimands, but is it about fitting present methods into the framework? Or is it about changing methods? My answer: some of each. Where different methods are needed, there are challenging problems in estimating some desirable estimands, but there may also be desirable estimands that can be estimated easily and robustly.

Trimmed means for symptom trials with dropouts.

Dropouts from randomized trials, often for lack of efficacy or toxicity, have usually been handled as 'missing data'. We suggest that they are instead complete observations, just not numeric ones. We propose an exact test of the hypothesis of no drug effect, taking all randomized patients into account, based on a readily interpretable statistic. The method also copes with a drug that is toxic in some patients but beneficial to others, a difficult problem for standard methods. A robust conclusion of efficacy can be drawn with no assumptions other than randomization. Published 2016. This article is a U.S. Government work and is in the public domain in the USA.

Missing data in clinical trials: control-based mean imputation and sensitivity analysis.

In some randomized (drug versus placebo) clinical trials, the estimand of interest is the between-treatment difference in population means of a clinical endpoint that is free from the confounding effects of "rescue" medication (e.g., HbA1c change from baseline at 24 weeks that would be observed without rescue medication regardless of whether or when the assigned treatment was discontinued). In such settings, a missing data problem arises if some patients prematurely discontinue from the trial or initiate rescue medication while in the trial, the latter necessitating the discarding of post-rescue data. We caution that the commonly used mixed-effects model repeated measures analysis with the embedded missing at random assumption can deliver an exaggerated estimate of the aforementioned estimand of interest. This happens, in part, due to implicit imputation of an overly optimistic mean for "dropouts" (i.e., patients with missing endpoint data of interest) in the drug arm. We propose an alternative approach in which the missing mean for the drug arm dropouts is explicitly replaced with either the estimated mean of the entire endpoint distribution under placebo (primary analysis) or a sequence of increasingly more conservative means within a tipping point framework (sensitivity analysis); patient-level imputation is not required. A supplemental "dropout = failure" analysis is considered in which a common poor outcome is imputed for all dropouts followed by a between-treatment comparison using quantile regression. All analyses address the same estimand and can adjust for baseline covariates. Three examples and simulation results are used to support our recommendations.

Sensitivity analysis for missing data in regulatory submissions.

The National Research Council Panel on Handling Missing Data in Clinical Trials recommended that sensitivity analyses have to be part of the primary reporting of findings from clinical trials. Their specific recommendations, however, seem not to have been taken up rapidly by sponsors of regulatory submissions. The NRC report's detailed suggestions are along rather different lines than what has been called sensitivity analysis in the regulatory setting up to now. Furthermore, the role of sensitivity analysis in regulatory decision-making, although discussed briefly in the NRC report, remains unclear. This paper will examine previous ideas of sensitivity analysis with a view to explaining how the NRC panel's recommendations are different and possibly better suited to coping with present problems of missing data in the regulatory setting. It will also discuss, in more detail than the NRC report, the relevance of sensitivity analysis to decision-making, both for applicants and for regulators. Published 2015. This article is a U.S. Government work and is in the public domain in the USA.

A taxonomy of estimands for regulatory clinical trials with discontinuations.

The National Research Council Panel on Handling Missing Data in Clinical Trials recommended that protocols for clinical trials 'explicitly define... causal estimands of primary interest'. In discussions with sponsors of clinical trials since the publication of the National Research Council report, the expression causal estimands has been the subject of confusion. It may not be entirely clear what the National Research Council panel meant, and in any case, it has not been clear how this recommendation might be put in practice. This paper's purpose is to say how the working group understands it and how we think it should be put in practice. We classify possible choices of estimand according to their usefulness for regulatory purposes in various clinical settings. Published 2015. This article is a U.S. Government work and is in the public domain in the USA.

A regulatory perspective on missing data in the aftermath of the NRC report.

The issuance of a report in 2010 by the National Research Council (NRC) of the National Academy of Sciences entitled 'The Prevention and Treatment of Missing Data in Clinical Trials,' commissioned by the US Food and Drug Administration, had an immediate impact on the way that statisticians and clinical researchers in both industry and regulatory agencies think about the missing data problem. We believe that there is currently great potential to improve study quality and interpretability-by reducing the amount of missing data through changes in trial design and conduct and by planning and conducting analyses that better account for the missing information. Here, we describe our view on some of the recommendations in the report and suggest ways in which these recommendations can be incorporated into new or ongoing clinical trials in order to improve their chance of success. Published 2015. This article is a U.S. Government work and is in the public domain in the USA.

Defining Treatment Effects: A Primer for Nonstatisticians.

It would often be of interest to know the effect of a drug compared to control in people who take the drug. However, different people will likely take the drug and the control. Thus, comparing takers of the drug to takers of the control does not yield a drug effect. Drug effects in drug takers can be estimated, but first they must be carefully defined.

Considerations for Developing Targeted Therapies in Low-Frequency Molecular Subsets of a Disease.

Advances in our understanding of the molecular underpinnings of disease have spurred the development of targeted therapies and the use of precision medicine approaches in patient care. While targeted therapies have improved our capability to provide effective treatments to patients, they also present additional challenges to drug development and benefit-risk assessment such as identifying the subset(s) of patients likely to respond to the drug, assessing heterogeneity in response across molecular subsets of a disease, and developing diagnostic tests to identify patients for treatment. These challenges are particularly difficult to address when targeted therapies are developed to treat diseases with multiple molecular subtypes that occur at low frequencies. To help address these challenges, the US Food and Drug Administration recently published a draft guidance entitled "Developing Targeted Therapies in Low-Frequency Molecular Subsets of a Disease." Here we provide additional information on specific aspects of targeted therapy development in diseases with low-frequency molecular subsets.

Probability models and computational models for ANOVA in multicenter clinical trials.

Multicenter trials are usually analyzed by fixed-effects, two-way analysis of variance. A mixed-effects computation with random treatment-by-center interaction may produce better results even if this interaction is considered truly to be a fixed effect. It may be more appropriate, however, to consider the treatment-by-center interaction to be a random effect, anyway.