Totality of evidence of the effectiveness of repurposed therapies for COVID-19: Can we use real-world studies alongside randomized controlled trials?

Rapid and robust strategies to evaluate the efficacy and effectiveness of novel and existing pharmacotherapeutic interventions (repurposed treatments) in future pandemics are required. Observational "real-world studies" (RWS) can report more quickly than randomized controlled trials (RCTs) and would have value were they to yield reliable results. Both RCTs and RWS were deployed during the coronavirus disease 2019 (COVID-19) pandemic. Comparing results between them offers a unique opportunity to determine the potential value and contribution of each. A learning review of these parallel evidence channels in COVID-19, based on quantitative modeling, can help improve speed and reliability in the evaluation of repurposed therapeutics in a future pandemic. Analysis of all-cause mortality data from 249 observational RWS and RCTs across eight treatment regimens for COVID-19 showed that RWS yield more heterogeneous results, and generally overestimate the effect size subsequently seen in RCTs. This is explained in part by a few study factors: the presence of RWS that are imbalanced for age, gender, and disease severity, and those reporting mortality at 2 weeks or less. Smaller studies of either type contributed negligibly. Analysis of evidence generated sequentially during the pandemic indicated that larger RCTs drive our ability to make conclusive decisions regarding clinical benefit of each treatment, with limited inference drawn from RWS. These results suggest that when evaluating therapies in future pandemics, (1) large RCTs, especially platform studies, be deployed early; (2) any RWS should be large and should have adequate matching of known confounders and long follow-up; (3) reporting standards and data standards for primary endpoints, explanatory factors, and key subgroups should be improved; in addition, (4) appropriate incentives should be in place to enable access to patient-level data; and (5) an overall aggregate view of all available results should be available at any given time.

Platform trials to evaluate the benefit-risk of COVID-19 therapeutics: Successes, learnings, and recommendations for future pandemics.

BACKGROUND: In response to the COVID-19 global pandemic, multiple platform trials were initiated to accelerate evidence generation of potential therapeutic interventions. Given a rapidly evolving and dynamic pandemic, platform trials have a key advantage over traditional randomized trials: multiple interventions can be investigated under a master protocol sharing a common infrastructure. METHODS: This paper focuses on nine platform trials that were instrumental in advancing care in COVID-19 in the hospital and community setting. A semi-structured qualitative interview was conducted with the principal investigators and lead statisticians of these trials. Information from the interviews and public sources were tabulated and summarized across trials, and recommendations for best practice for the next health crisis are provided. RESULTS: Based on the information gathered takeaways were identified as 1) the existence of some aspect of trial design or conduct (e.g., existing network of investigators or colleagues, infrastructure for data capture and relevant statistical expertise) was a key success factor; 2) the choice of treatments (e.g., repurposed drugs) had major impact on the trials as did the choice of primary endpoint; and 3) the lack of coordination across trials was flagged as an area for improvement. CONCLUSION: These trials deployed during the COVID-19 pandemic demonstrate how to achieve both speed and quality of evidence generation regarding clinical benefit (or not) of existing therapies to treat new pathogens in a pandemic setting. As a group, these trials identified treatments that worked, and many that did not, in a matter of months.

A Tutorial on Modern Bayesian Methods in Clinical Trials.

Clinical trials continue to be the gold standard for evaluating new medical technologies. New advancements in modern computation power have led to increasing interest in Bayesian methods. Despite the multiple benefits of Bayesian approaches, application to clinical trials has been limited. Based on insights from the survey of clinical researchers in drug development conducted by the Drug Information Association Bayesian Scientific Working Group (DIA BSWG), insufficient knowledge of Bayesian approaches was ranked as the most important perceived barrier to implementing Bayesian methods. Results of the same survey indicate that clinical researchers may find the interpretation of results from a Bayesian analysis to be more useful than conventional interpretations. In this article, we illustrate key concepts tied to Bayesian methods, starting with familiar concepts widely used in clinical practice before advancing in complexity, and use practical illustrations from clinical development.

Bayesian Strategies in Rare Diseases.

Bayesian strategies for planning and analyzing clinical trials have become a viable choice, especially in rare diseases where drug development faces many challenges and stakeholders are interested in innovations that may help overcome them. Disease natural history and clinical outcomes occurrence and variability are often poorly understood. Standard trial designs are not optimized to obtain adequate safety and efficacy data from small numbers of patients. Bayesian methods are well-suited for adaptive trials, with an accelerated learning curve. Using Bayesian statistics can be advantageous in that design choices and their consequences are considered carefully, continuously monitored, and updated where necessary, which ultimately provides a natural and principled way of seamlessly combining prior clinical information with data, within a solid decision theoretical framework. In this article, we introduce the Bayesian option in the rare disease context to support clinical decision-makers in selecting the best choice for their drug development project. Many researchers in drug development show reluctance to using Bayesian statistics, and the top-two reported barriers are insufficient knowledge of Bayesian approaches and a lack of clarity or guidance from regulators. Here we introduce concepts of borrowing, extrapolation, adaptation, and modeling and illustrate them with examples that have been discussed or developed with regulatory bodies to show how Bayesian strategies can be applied to drug development in rare diseases.

U.S. Food and Drug Administration Reasoning in Approval Decisions When Efficacy Evidence Is Borderline, 2013-2018.

BACKGROUND: The U.S. Food and Drug Administration (FDA) has substantial flexibility in its approval criteria in the context of life-threatening disease and unmet therapeutic need. OBJECTIVE: To understand the FDA's evidentiary standards when flexible criteria are employed. DESIGN: Case series. SETTING: Applications submitted between 2013 and 2018 that went through multiple review cycles because the evidence for clinical efficacy was initially deemed insufficient. MEASUREMENTS: Information was obtained from the approval package (available on Drugs@FDA), including advisory committee minutes, FDA reviews, and complete response letters. RESULTS: Of 912 applications reviewed, 117 went through multiple review cycles; only 22 of these faced additional review primarily because of issues related to clinical efficacy. Concerns about the end point, the clinical meaningfulness of the observed effect, and inconsistent results were common bases for initial rejection. In 7 of the 22 cases, the approval did not require new evidence but rather new interpretations of the original evidence. No FDA decisions cited reasoning used in previous decisions. LIMITATION: The conclusions rely on the authors' interpretation of the FDA statements and on a series of "close calls." CONCLUSION: The FDA has no mechanism to find or tradition to cite similar cases when weighing evidence for approvals, resulting in standalone, bespoke decisions. These decisions show highly variable criteria for "substantial evidence" when flexible evidential criteria are used, highlighted by the recent approval of aducanumab. A precedential tradition and suitable information system are required for the FDA to improve institutional memory and build upon past decisions. These would increase the FDA's decisional transparency, consistency, and predictability, which are critical to preserving the FDA's most valuable asset, the public's trust. PRIMARY FUNDING SOURCE: U.S. Food and Drug Administration.

Vaccines After an Emergency Use Authorization (EUA): Modern Evidence Generation Approaches.

Every medical product requires additional study even after regulatory approval. We highlight several lines of enquiry to advance our understanding of COVID19 vaccines post authorization: identifying key population segments warranting more study, assessment of efficacy, and of safety data, harmonization of data relating to immune response and developing mechanisms for data and knowledge sharing across countries. We show how innovative trial designs and sources from real world data play a critical role in generating evidence.

Sources of Safety Data and Statistical Strategies for Design and Analysis: Real World Insights.

BACKGROUND: Although randomized controlled clinical trials provide necessary information and serve as the basis for regulatory decision making, a significant gap exists between the evidence these trials provide and what the biomedical community needs. It is recognized that a wealth of data are routinely collected outside clinical trials. Such real-world data (RWD) are not of comparable quality, it does not have similar immunity from bias and confounding as data collected in randomized clinical trials, but it might offer additional understanding of the benefit-risk, provide new insights to different stakeholders, and aid in regulatory decision making. This can be especially true when rare but serious adverse events are considered because randomized clinical trials are often not large enough and have insufficient duration to address safety concerns fully. Also, the passage of the 21st Century Cures bill passed by Congress in 2016 means that several data sources outside traditional clinical trials will play a greater role in regulatory decision making. This manuscript is third in a series of articles from the American Statistical Association Biopharmaceutical Section Safety Working Group. METHODS: In this manuscript, authors reviewed some RWD sources and shared considerations for statistical strategies and methodologies needed to design and analyze observational safety studies and pragmatic trials. RESULTS: Authors presented case studies and shared recommendations for statistical methods necessary to design and analyze safety trials using RWD. CONCLUSIONS: RWD is an important source of safety data that contribute to the totality of safety information available to generate evidence for regulators, sponsors, payers, physicians, and patients. However, it is important to determine if such data are fit for purpose.

Sources of Safety Data and Statistical Strategies for Design and Analysis: Transforming Data Into Evidence.

BACKGROUND: Safety evaluation is a key aspect of medical product development. It is a continual and iterative process requiring thorough thinking, and dedicated time and resources. METHODS: In this article, we discuss how safety data are transformed into evidence to establish and refine the safety profile of a medical product, and how the focus of safety evaluation, data sources, and statistical methods change throughout a medical product's life cycle. RESULTS: Some challenges and statistical strategies for medical product safety evaluation are discussed. Examples of safety issues identified in different periods, that is, premarketing and postmarketing, are discussed to illustrate how different sources are used in the safety signal identification and the iterative process of safety assessment. The examples highlighted range from commonly used pediatric vaccine given to healthy children to medical products primarily used to treat a medical condition in adults. These case studies illustrate that different products may require different approaches, and once a signal is discovered, it could impact future safety assessments. CONCLUSIONS: Many challenges still remain in this area despite advances in methodologies, infrastructure, public awareness, international harmonization, and regulatory enforcement. Innovations in safety assessment methodologies are pressing in order to make the medical product development process more efficient and effective, and the assessment of medical product marketing approval more streamlined and structured. Health care payers, providers, and patients may have different perspectives when weighing in on clinical, financial and personal needs when therapies are being evaluated.

Logistic Regression Likelihood Ratio Test Analysis for Detecting Signals of Adverse Events in Post-market Safety Surveillance.

The Vaccine Adverse Event Reporting System (VAERS) and other product surveillance systems compile reports of product-associated adverse events (AEs), and these reports may include a wide range of information including age, gender, and concomitant vaccines. Controlling for possible confounding variables such as these is an important task when utilizing surveillance systems to monitor post-market product safety. A common method for handling possible confounders is to compare observed product-AE combinations with adjusted baseline frequencies where the adjustments are made by stratifying on observable characteristics. Though approaches such as these have proven to be useful, in this article we propose a more flexible logistic regression approach which allows for covariates of all types rather than relying solely on stratification. Indeed, a main advantage of our approach is that the general regression framework provides flexibility to incorporate additional information such as demographic factors and concomitant vaccines. As part of our covariate-adjusted method, we outline a procedure for signal detection that accounts for multiple comparisons and controls the overall Type 1 error rate. To demonstrate the effectiveness of our approach, we illustrate our method with an example involving febrile convulsion, and we further evaluate its performance in a series of simulation studies.

Checking distributional assumptions for pharmacokinetic summary statistics based on simulations with compartmental models.

Bioequivalence (BE) studies are an essential part of the evaluation of generic drugs. The most common in vivo BE study design is the two-period two-treatment crossover design. AUC (area under the concentration-time curve) and Cmax (maximum concentration) are obtained from the observed concentration-time profiles for each subject from each treatment under each sequence. In the BE evaluation of pharmacokinetic crossover studies, the normality of the univariate response variable, e.g. log(AUC)1 or log(Cmax), is often assumed in the literature without much evidence. Therefore, we investigate the distributional assumption of the normality of response variables, log(AUC) and log(Cmax), by simulating concentration-time profiles from two-stage pharmacokinetic models (commonly used in pharmacokinetic research) for a wide range of pharmacokinetic parameters and measurement error structures. Our simulations show that, under reasonable distributional assumptions on the pharmacokinetic parameters, log(AUC) has heavy tails and log(Cmax) is skewed. Sensitivity analyses are conducted to investigate how the distribution of the standardized log(AUC) (or the standardized log(Cmax)) for a large number of simulated subjects deviates from normality if distributions of errors in the pharmacokinetic model for plasma concentrations deviate from normality and if the plasma concentration can be described by different compartmental models.

Seeking harmony: estimands and sensitivity analyses for confirmatory clinical trials.

In October 2014, the Steering Committee of the International Conference on Harmonization endorsed the formation of an expert working group to develop an addendum to the International Conference on Harmonization E9 guideline ("Statistical Principles for Clinical Trials"). The addendum will focus on two topics involving randomized confirmatory clinical trials: estimands and sensitivity analyses. Both topics are motivated, in part, by the need to improve the precision with which scientific questions of interest are formulated and addressed by clinical trialists and regulators, specifically in the context of post-randomization events such as use of rescue medication or missing data resulting from dropouts. Given the importance of these topics for the statistical and medical community, we articulate the reasons for the planned addendum. The resulting "ICH E9/R1" guideline will include a framework for improved trial planning, conduct, analysis, and interpretation; a draft is expected to be ready for public comment in the second half of 2016.

CBER's Experience With Adaptive Design Clinical Trials.

There is considerable interest among pharmaceutical and other medical product developers in adaptive clinical trials, in which knowledge learned during the course of a trial affects ongoing conduct or analysis of the trial. When the FDA released a draft Guidance document on adaptive design clinical trials in early 2010, expectations were high that it would lead to an increase in regulatory submissions involving adaptive design features, particularly for confirmatory trials. A 6-year (2008-2013) retrospective survey was performed within the Center for Biologics Evaluation and Research (CBER) at the FDA to gather information regarding the submission and evaluation of adaptive design trial proposals. We present an up-to-date summary of adaptive design proposals seen in CBER and provide an overview of our experiences. We share our concerns regarding the statistical issues and operational challenges raised during the review process for adaptive design trials. We also provide general recommendations for developing proposals for such trials. Our motivation in writing this paper was to encourage the best study design proposals to be submitted to CBER. Sometimes these can be adaptive, and sometimes a simpler design is most efficient.

Quantifying the impact of time-varying baseline risk adjustment in the self-controlled risk interval design.

PURPOSE: The self-controlled risk interval design is commonly used to assess the association between an acute exposure and an adverse event of interest, implicitly adjusting for fixed, non-time-varying covariates. Explicit adjustment needs to be made for time-varying covariates, for example, age in young children. It can be performed via either a fixed or random adjustment. The random-adjustment approach can provide valid point and interval estimates but requires access to individual-level data for an unexposed baseline sample. The fixed-adjustment approach does not have this requirement and will provide a valid point estimate but may underestimate the variance. We conducted a comprehensive simulation study to evaluate their performance. METHODS: We designed the simulation study using empirical data from the Food and Drug Administration-sponsored Mini-Sentinel Post-licensure Rapid Immunization Safety Monitoring Rotavirus Vaccines and Intussusception study in children 5-36.9 weeks of age. The time-varying confounder is age. We considered a variety of design parameters including sample size, relative risk, time-varying baseline risks, and risk interval length. RESULTS: The random-adjustment approach has very good performance in almost all considered settings. The fixed-adjustment approach can be used as a good alternative when the number of events used to estimate the time-varying baseline risks is at least the number of events used to estimate the relative risk, which is almost always the case. CONCLUSIONS: We successfully identified settings in which the fixed-adjustment approach can be used as a good alternative and provided guidelines on the selection and implementation of appropriate analyses for the self-controlled risk interval design.

Letter to the editor by the authors of Exact Calculation of Power and Sample Size in Bioequivalence Studies Using Two One-sided Tests, Pharmaceutical Statistics, DOI: 10.1002/pst.1666.

This article reflects the views of the authors and should not be construed to be those of the US Food and Drug Administration.

National Institutes of Health Consensus Development Project on Criteria for Clinical Trials in Chronic Graft-versus-Host Disease: III. The 2014 Biomarker Working Group Report.

Biology-based markers to confirm or aid in the diagnosis or prognosis of chronic graft-versus-host disease (GVHD) after allogeneic hematopoietic cell transplantation or monitor its progression are critically needed to facilitate evaluation of new therapies. Biomarkers have been defined as any characteristic that is objectively measured and evaluated as an indicator of a normal biological or pathogenic process, or of a pharmacologic response to a therapeutic intervention. Applications of biomarkers in chronic GVHD clinical trials or patient management include the following: (1) diagnosis and assessment of chronic GVHD disease activity, including distinguishing irreversible damage from continued disease activity; (2) prognostic risk to develop chronic GVHD; and (3) prediction of response to therapy. Sample collection for chronic GVHD biomarkers studies should be well documented following established quality control guidelines for sample acquisition, processing, preservation, and testing, at intervals that are both calendar and event driven. The consistent therapeutic treatment of subjects and standardized documentation needed to support biomarker studies are most likely to be provided in prospective clinical trials. To date, no chronic GVHD biomarkers have been qualified for use in clinical applications. Since our previous chronic GVHD Biomarkers Working Group report in 2005, an increasing number of chronic GVHD candidate biomarkers are available for further investigation. This paper provides a 4-part framework for biomarker investigations: identification, verification, qualification, and application with terminology based on Food and Drug Administration and European Medicines Agency guidelines.

Exact calculation of power and sample size in bioequivalence studies using two one-sided tests.

The number of subjects in a pharmacokinetic two-period two-treatment crossover bioequivalence study is typically small, most often less than 60. The most common approach to testing for bioequivalence is the two one-sided tests procedure. No explicit mathematical formula for the power function in the context of the two one-sided tests procedure exists in the statistical literature, although the exact power based on Owen's special case of bivariate noncentral t-distribution has been tabulated and graphed. Several approximations have previously been published for the probability of rejection in the two one-sided tests procedure for crossover bioequivalence studies. These approximations and associated sample size formulas are reviewed in this article and compared for various parameter combinations with exact power formulas derived here, which are computed analytically as univariate integrals and which have been validated by Monte Carlo simulations. The exact formulas for power and sample size are shown to improve markedly in realistic parameter settings over the previous approximations.

Preclinical safety evaluation of human platelets treated with antimicrobial peptides in severe combined immunodeficient mice.

BACKGROUND: Bacterial sepsis is a complication attributed to room temperature (RT)-stored platelets (PLTs) in transfusion medicine. Antimicrobial peptides (AMPs) are emerging as new therapeutic agents against microbes. We had previously demonstrated bactericidal activity of select synthetic AMPs against six types of bacteria in stored PLTs. In this report, we tested these AMPs for their potential antibody response and interference with the recovery and survival of human PLTs in an animal model. STUDY DESIGN AND METHODS: Two separate studies were conducted to evaluate the safety of the synthetic AMPs. 1) Two AMPs (PD3 and PD4), derived from thrombin-induced human PLT microbicidal protein, and four repeats of arginine-tryptophan (RW), containing two to five repeats (RW2-RW5), were tested in rabbits for potential antibody response. 2) RT-stored human PLTs treated for 2 hours with each of the six AMPs individually or with phosphate-buffered saline (PBS) alone were infused into severe combined immunodeficient (SCID) mice to evaluate their in vivo recovery and survival by flow cytometry. RESULTS: Except for PD3, which showed a weak immune response, all other peptides did not induce any detectable antibodies in rabbits. Furthermore, all six AMPs tested did not significantly affect the in vivo recovery and survival of human PLTs in SCID mice compared to PBS alone-treated PLTs. CONCLUSION: Preclinical evaluation studies reported here demonstrate that the selected AMPs used in the study did not adversely affect the human PLT recovery and survival in the SCID mouse model, suggesting further study of AMPs toward addressing the bacterial contamination of PLTs.

Multiple in silico tools predict phenotypic manifestations in congenital thrombotic thrombocytopenic purpura.

Congenital thrombotic thrombocytopenic purpura (cTTP) is a rare, recessively inherited genetic disorder with varying clinical presentation that is caused by ADAMTS13 mutations. Several studies have found limited associations between ADAMTS13 mutations and cTTP phenotype. The use of in silico tools that examine multiple mutation characteristics may better predict phenotype. We analysed 118 ADAMTS13 mutations found in 144 cTTP patients reported in the literature and examined associations of several mutation characteristics, including N-terminal proximity, the evolutionary conservation of the affected amino acid position, as well as amino acid charge/phosphorylation and genetic codon usage to disease phenotype. Structure-altering mutations were examined for their impact on ADAMTS13 function based on existing ADAMTS13 crystallographic data (AA 77-685). Our in silico data indicate that: (i) The position of the mutation in the N- or C-terminus, (ii) evolutionary conservation and (iii) codon usage of the affected mutation position are associated with disease parameters, such as age of onset, organ damage and fresh frozen plasma prophylaxis. In conclusion, the usage of multiple in silico tools presents a promising strategy in refining predictions for the diverse presentation of cTTP. Enhancing our utilization of in silico tools to find genotype-phenotype associations will create better-tailored approaches for individual patient treatment.