The Acute Stroke Therapy by Inhibition of Neutrophils study - Key features and impact.

The Acute Stroke Therapy by Inhibition of Neutrophils (ASTIN) study, initiated in November of the year 2000, is now widely recognized as having been a landmark study in the history of clinical trials. We look at why this is the case by considering its key features and impact. These key features are: the use of Bayesian design and analysis; the use of the normal dynamic linear model; the response adaptive nature of the study; the use of real-time dosing decisions; and the use of an integrated model to predict 90-day response on the Scandinavian Stroke Scale. Our overall conclusion is that the ASTIN study's main impact came from showing the clinical trial community the feasibility of the novel design and analysis used when most of these key features were rarely used in industry trials, let alone used together in one trial in a disease area with a tremendous unmet medical need.

Presenting Risks and Benefits: Helping the Data Monitoring Committee Do Its Job.

Data monitoring committees (DMCs), or data and safety monitoring boards, protect clinical trial participants by conducting benefit-risk assessments during the course of a clinical trial. These evaluations may be improved by broader access to data and more effective analyses and presentation. Data monitoring committees should have access to all data, including efficacy data, at each interim review. The DMC reports should include graphical presentations that summarize benefits and harms in efficient ways. Benefit-risk assessments should include summaries that are consistent with the intention-to-treat principle and have a pragmatic focus. This article provides examples of graphical summaries that integrate benefits and harms, and proposes that such summaries become standard in DMC reports.

Lessons from meta-analyses of randomized clinical trials for analysis of distributed networks of observational databases.

Networks of constellations of longitudinal observational databases, often electronic medical records or transactional insurance claims or both, are increasingly being used for studying the effects of medicinal products in real-world use. Such databases are frequently configured as distributed networks. That is, patient-level data are kept behind firewalls and not communicated outside of the data vendor other than in aggregate form. Instead, data are standardized across the network, and queries of the network are executed locally by data partners, and summary results provided to a central research partner(s) for amalgamation, aggregation, and summarization. Such networks can be huge covering years of data on upwards of 100 million patients. Examples of such networks include the FDA Sentinel Network, ASPEN, CNODES, and EU-ADR. As this is a new emerging field, we note in this paper the conceptual similarities and differences between the analysis of distributed networks and the now well-established field of meta-analysis of randomized clinical trials (RCTs). We recommend, wherever appropriate, to apply learnings from meta-analysis to help guide the development of distributed network analyses of longitudinal observational databases.

Changes are still needed on multiple co-primary endpoints.

The Food and Drug Administration in the United States issued a much-awaited draft guidance on 'Multiple Endpoints in Clinical Trials' in January 2017. The draft guidance is well written and contains consistent message on the technical implementation of the principles laid out in the guidance. In this commentary, we raise a question on applying the principles to studies designed from a safety perspective. We then direct our attention to issues related to multiple co-primary endpoints. In a paper published in the Drug Information Journal in 2007, Offen et al. give examples of disorders where multiple co-primary endpoints are required by regulators. The standard test for multiple co-primary endpoints is the min test which tests each endpoint individually, at the one-sided 2.5% level, for a confirmatory trial. This approach leads to a substantial loss of power when the number of co-primary endpoints exceeds 2, a fact acknowledged in the draft guidance. We review approaches that have been proposed to tackle the problem of power loss and propose a new one. Using recommendations by Chen et al. for the assessment of drugs for vulvar and vaginal atrophy published in the Drug Information Journal in 2010, we argue the need for more changes and urge a path forward that uses different levels of claims to reflect the effectiveness of a product on multiple endpoints that are equally important and mostly unrelated. Copyright © 2017 John Wiley & Sons, Ltd.

A comparison of five approaches to decision-making for a first clinical trial of efficacy.

The first trial of clinical efficacy is an important step in the development of a compound. Such a trial gives the first indication of whether a compound is likely to have the efficacy needed to be successful. Good decisions dictate that good compounds have a large probability of being progressed and poor compounds have a large probability of being stopped. In this paper, we consider and contrast five approaches to decision-making that have been used. To illustrate the use of the five approaches, we conduct a comparison for two plausible scenarios with associated assumptions for sample sizing. The comparison shows some large differences in performance characteristics of the different procedures. Which decision-making procedures and associated performance characteristics are preferred will depend on the focus of interest and the decision maker's attitude to risk. Copyright © 2016 John Wiley & Sons, Ltd.

Implementation of AMNOG: An industry perspective.

In 2010, the Federal Parliament (Bundestag) of Germany passed a new law (Arzneimittelmarktneuordnungsgesetz, AMNOG) on the regulation of medicinal products that applies to all pharmaceutical products with active ingredients that are launched beginning January 1, 2011. The law describes the process to determine the price at which an approved new product will be reimbursed by the statutory health insurance system. The process consists of two phases. The first phase assesses the additional benefit of the new product versus an appropriate comparator (zweckmäßige Vergleichstherapie, zVT). The second phase involves price negotiation. Focusing on the first phase, this paper investigates requirements of benefit assessment of a new product under this law with special attention on the methods applied by the German authorities on issues such as the choice of the comparator, patient relevant endpoints, subgroup analyses, extent of benefit, determination of net benefit, primary and secondary endpoints, and uncertainty of the additional benefit. We propose alternative approaches to address the requirements in some cases and invite other researchers to help develop solutions in other cases.

Desirability of Outcome Ranking (DOOR) and Response Adjusted for Duration of Antibiotic Risk (RADAR).

Clinical trials that compare strategies to optimize antibiotic use are of critical importance but are limited by competing risks that distort outcome interpretation, complexities of noninferiority trials, large sample sizes, and inadequate evaluation of benefits and harms at the patient level. The Antibacterial Resistance Leadership Group strives to overcome these challenges through innovative trial design. Response adjusted for duration of antibiotic risk (RADAR) is a novel methodology utilizing a superiority design and a 2-step process: (1) categorizing patients into an overall clinical outcome (based on benefits and harms), and (2) ranking patients with respect to a desirability of outcome ranking (DOOR). DOORs are constructed by assigning higher ranks to patients with (1) better overall clinical outcomes and (2) shorter durations of antibiotic use for similar overall clinical outcomes. DOOR distributions are compared between antibiotic use strategies. The probability that a randomly selected patient will have a better DOOR if assigned to the new strategy is estimated. DOOR/RADAR represents a new paradigm in assessing the risks and benefits of new strategies to optimize antibiotic use.

The shrinking or disappearing observed treatment effect.

It is frequently noted that an initial clinical trial finding was not reproduced in a later trial. This is often met with some surprise. Yet, there is a relatively straightforward reason partially responsible for this observation. In this article, we examine this reason by first reviewing some findings in a recent publication in the Journal of the American Medical Association. To help explain the non-negligible chance of failing to reproduce a previous positive finding, we compare a series of trials to successive diagnostic tests used for identifying a condition. To help explain the suspicion that the treatment effect, when observed in a subsequent trial, seems to have decreased in magnitude, we draw a conceptual analogy between phases II-III development stages and interim analyses of a trial with a group sequential design. Both analogies remind us that what we observed in an early trial could be a false positive or a random high. We discuss statistical sources for these occurrences and discuss why it is important for statisticians to take these into consideration when designing and interpreting trial results.

Subgroup analyses in cost-effectiveness analyses to support health technology assessments.

'Success' in drug development is bringing to patients a new medicine that has an acceptable benefit-risk profile and that is also cost-effective. Cost-effectiveness means that the incremental clinical benefit is deemed worth paying for by a healthcare system, and it has an important role in enabling manufacturers to obtain new medicines to patients as soon as possible following regulatory approval. Subgroup analyses are increasingly being utilised by decision-makers in the determination of the cost-effectiveness of new medicines when making recommendations. This paper highlights the statistical considerations when using subgroup analyses to support cost-effectiveness for a health technology assessment. The key principles recommended for subgroup analyses supporting clinical effectiveness published by Paget et al. are evaluated with respect to subgroup analyses supporting cost-effectiveness. A health technology assessment case study is included to highlight the importance of subgroup analyses when incorporated into cost-effectiveness analyses. In summary, we recommend planning subgroup analyses for cost-effectiveness analyses early in the drug development process and adhering to good statistical principles when using subgroup analyses in this context. In particular, we consider it important to provide transparency in how subgroups are defined, be able to demonstrate the robustness of the subgroup results and be able to quantify the uncertainty in the subgroup analyses of cost-effectiveness.

The practice of pre-marketing safety assessment in drug development.

The last 15 years have seen a substantial increase in efforts devoted to safety assessment by statisticians in the pharmaceutical industry. While some of these efforts were driven by regulations and public demand for safer products, much of the motivation came from the realization that there is a strong need for a systematic approach to safety planning, evaluation, and reporting at the program level throughout the drug development life cycle. An efficient process can help us identify safety signals early and afford us the opportunity to develop effective risk minimization plan early in the development cycle. This awareness has led many pharmaceutical sponsors to set up internal systems and structures to effectively conduct safety assessment at all levels (patient, study, and program). In addition to process, tools have emerged that are designed to enhance data review and pattern recognition. In this paper, we describe advancements in the practice of safety assessment during the premarketing phase of drug development. In particular, we share examples of safety assessment practice at our respective companies, some of which are based on recommendations from industry-initiated working groups on best practice in recent years.

Designing studies to find early signals of efficacy.

We introduce the idea of a design to detect signals of efficacy in early phase clinical trials. Such a design features three possible decisions: to kill the compound; to continue with staged development; or to continue with accelerated development of the compound. We describe how such studies improve the trade-off between the two errors of killing a compound with good efficacy and committing to a complete full development program for a compound that has no efficacy and describe how they can be designed. We argue that such studies could be used to screen compounds at the proof-of-concept state, reduce late Phase 2 attrition, and speed up the development of highly efficacious drugs.

A new proposal for randomized start design to investigate disease-modifying therapies for Alzheimer disease.

BACKGROUND: The increasing prevalence of Alzheimer disease (AD) and lack of effective agents to attenuate progression have accelerated research and development of disease modifying (DM) therapies. The traditional parallel group design and single time point analysis used in the support of past AD drug approvals address symptomatic benefit over relatively short treatment durations. More recent trials investigating disease modification are by necessity longer in duration and require larger sample sizes. Nevertheless, trial design and analysis remain mostly unchanged and may not be adequate to meet the objective of demonstrating disease modification. Randomized start design (RSD) has been proposed as an option to study DM effects, but its application in AD trials may have been hampered by certain methodological challenges. PURPOSE: To address the methodological issues that have impeded more extensive use of RSD in AD trial and to encourage other researchers to develop novel design and analysis methodologies to better ascertain DM effects for the next generation of AD therapies, we propose a stepwise testing procedure to evaluate potential DM effects of novel AD therapies. METHODS: Alzheimer Disease Assessment Scale-Cognitive Subscale (ADAS-cog) is used for illustration. We propose to test three hypotheses in a stepwise sequence. The three tests pertain to treatment difference at two separate time points and a difference in the rate of change. Estimation is facilitated by the Mixed-effects Model for Repeated Measures approach. The required sample size is estimated using Monte Carlo simulations and by modeling ADAS-cog data from prior longitudinal AD studies. RESULTS: The greatest advantage of the RSD proposed in this article is its ability to critically address the question on a DM effect. The AD trial using the new approach would be longer (12-month placebo period plus 12-month delay-start period; total 24-month duration) and require more subjects (about 1000 subjects per arm for the non-inferiority margin chosen in the illustration). It would also require additional evaluations to estimate the rate of ADAS-cog change toward the end of the trial. LIMITATIONS: A regulatory claim of disease modification for any compound will likely require additional verification of a drug's effect on a validated biomarker of Alzheimer's pathology. CONCLUSIONS: Incorporation of the RSD in AD trials is feasible. With proper trial setup and statistical procedures, this design could support the detection of a disease-modifying effect. In our opinion, a two-phase RSD with a stepwise hypothesis testing procedure could be a reasonable option for future studies.

Reporting cumulative proportion of subjects with an adverse event based on data from multiple studies.

Experience has shown us that when data are pooled from multiple studies to create an integrated summary, an analysis based on naïvely-pooled data is vulnerable to the mischief of Simpson's Paradox. Using the proportions of patients with a target adverse event (AE) as an example, we demonstrate the Paradox's effect on both the comparison and the estimation of the proportions. While meta analytic approaches have been recommended and increasingly used for comparing safety data between treatments, reporting proportions of subjects experiencing a target AE based on data from multiple studies has received little attention. In this paper, we suggest two possible approaches to report these cumulative proportions. In addition, we urge that regulatory guidelines on reporting such proportions be established so that risks can be communicated in a scientifically defensible and balanced manner.

The role of the minimum clinically important difference and its impact on designing a trial.

The minimum clinically important difference (MCID) between treatments is recognized as a key concept in the design and interpretation of results from a clinical trial. Yet even assuming such a difference can be derived, it is not necessarily clear how it should be used. In this paper, we consider three possible roles for the MCID. They are: (1) using the MCID to determine the required sample size so that the trial has a pre-specified statistical power to conclude a significant treatment effect when the treatment effect is equal to the MCID; (2) requiring with high probability, the observed treatment effect in a trial, in addition to being statistically significant, to be at least as large as the MCID; (3) demonstrating via hypothesis testing that the effect of the new treatment is at least as large as the MCID. We will examine the implications of the three different possible roles of the MCID on sample size, expectations of a new treatment, and the chance for a successful trial. We also give our opinion on how the MCID should generally be used in the design and interpretation of results from a clinical trial.

Subgroup analyses of clinical effectiveness to support health technology assessments.

Subgroup analysis is an integral part of access and reimbursement dossiers, in particular health technology assessment (HTA), and their HTA recommendations are often limited to subpopulations. HTA recommendations for subpopulations are not always clear and without controversies. In this paper, we review several HTA guidelines regarding subgroup analyses. We describe good statistical principles for subgroup analyses of clinical effectiveness to support HTAs and include case examples where HTA recommendations were given to subpopulations only. Unlike regulatory submissions, pharmaceutical statisticians in most companies have had limited involvement in the planning, design and preparation of HTA/payers submissions. We hope to change this by highlighting how pharmaceutical statisticians should contribute to payers' submissions. This includes early engagement in reimbursement strategy discussions to influence the design, analysis and interpretation of phase III randomized clinical trials as well as meta-analyses/network meta-analyses. The focus on this paper is on subgroup analyses relating to clinical effectiveness as we believe this is the first key step of statistical involvement and influence in the preparation of HTA and reimbursement submissions.

FDA draft guidance on adaptive design clinical trials: Pfizer's perspective.

The Food and Drug Administration of the United States issued a draft guidance on adaptive design clinical trials in February 2010. This draft guidance has attracted a lot of attention because of the increasing interest in adaptive trials by the pharmaceutical industry in recent years. In this paper, we report on highlights of comments collected within Pfizer on this draft guidance. In addition, we share Pfizer's internal journey to promote efficient trial designs since 2005. Adaptive designs have been part of that journey.

Determining a minimum clinically important difference between treatments for a patient-reported outcome.

Patient-reported outcomes are important for assessing the effectiveness of treatments in many disease areas. For this reason, many new instruments that capture patient-reported outcomes have been developed over the past several decades. With the development of each new instrument, there is the ensuing question of what constitutes a minimum clinically important difference between treatments when using the new instrument. In this paper we describe a method for estimating a minimum clinically important difference between treatments for a patient-reported outcome through a desired difference in response rates for a definition of a responder. As well as being of interest in its own right, the use of a minimum clinically important difference on the patient-reported outcome scale is likely to lead to sample size advantages. We illustrate the method with data on neuropathic pain when responder is defined by requiring at least some improvement in the Patient Global Impression of Change and when responder is defined by existing responder definitions.

Viewpoints on the FDA draft adaptive designs guidance from the PhRMA working group.

The US Food and Drug Administration has recently released a draft guidance document on adaptive clinical trials. We comment on the document from the particular perspective of the authors as members of a PhRMA working group on this topic, which has interacted with FDA personnel on adaptive trial issue during recent years. We describe the activities and prior work of our working group, and use this as a basis to discuss the content of the guidance document as it relates to several issues of current relevance, such as data monitoring processes, adaptive dose finding, so-called seamless trial designs, and sample size reestimation.