Accelerating drug development at Bristol Myers Squibb through innovation.

This paper focuses on the use of novel technologies and innovative trial designs to accelerate evidence generation and increase pharmaceutical Research and Development (R&D) productivity, at Bristol Myers Squibb. We summarize learnings with case examples, on how we prepared and continuously evolved to address the increasing cost, complexities, and external pressures in drug development, to bring innovative medicines to patients much faster. These learnings were based on review of internal efforts toward accelerating R&D focusing on four key areas: adopting innovative trial designs, optimizing trial designs, leveraging external control data, and implementing novel methods using artificial intelligence and machine learning.

Platform trials to evaluate the benefit-risk of COVID-19 therapeutics: Successes, learnings, and recommendations for future pandemics.

BACKGROUND: In response to the COVID-19 global pandemic, multiple platform trials were initiated to accelerate evidence generation of potential therapeutic interventions. Given a rapidly evolving and dynamic pandemic, platform trials have a key advantage over traditional randomized trials: multiple interventions can be investigated under a master protocol sharing a common infrastructure. METHODS: This paper focuses on nine platform trials that were instrumental in advancing care in COVID-19 in the hospital and community setting. A semi-structured qualitative interview was conducted with the principal investigators and lead statisticians of these trials. Information from the interviews and public sources were tabulated and summarized across trials, and recommendations for best practice for the next health crisis are provided. RESULTS: Based on the information gathered takeaways were identified as 1) the existence of some aspect of trial design or conduct (e.g., existing network of investigators or colleagues, infrastructure for data capture and relevant statistical expertise) was a key success factor; 2) the choice of treatments (e.g., repurposed drugs) had major impact on the trials as did the choice of primary endpoint; and 3) the lack of coordination across trials was flagged as an area for improvement. CONCLUSION: These trials deployed during the COVID-19 pandemic demonstrate how to achieve both speed and quality of evidence generation regarding clinical benefit (or not) of existing therapies to treat new pathogens in a pandemic setting. As a group, these trials identified treatments that worked, and many that did not, in a matter of months.

From Meaningful Outcomes to Meaningful Change Thresholds: A Path to Progress for Establishing Digital Endpoints.

This paper examines the use of digital endpoints (DEs) derived from digital health technologies (DHTs), focusing primarily on the specific considerations regarding the determination of meaningful change thresholds (MCT). Using DHTs in drug development is becoming more commonplace. There is general acceptance of the value of DHTs supporting patient-centric trial design, capturing data outside the traditional clinical trial setting, and generating DEs with the potential to be more sensitive to change than conventional assessments. However, the transition from exploratory endpoints to primary and secondary endpoints capable of supporting labeling claims requires these endpoints to be substantive with reproducible population-specific values. Meaningful change represents the amount of change in an endpoint measure perceived as important to patients and should be determined for each digital endpoint and given population under consideration. This paper examines existing approaches to determine meaningful change thresholds and explores examples of these methodologies and their use as part of DE development: emphasizing the importance of determining what aspects of health are important to patients and ensuring the DE captures these concepts of interest and aligns with the overarching endpoint strategy. Examples are drawn from published DE qualification documentation and responses to qualification submissions under review by the various regulatory authorities. It is the hope that these insights will inform and strengthen the development and validation of DEs as drug development tools, particularly for those new to the approaches to determine MCTs.

Real-World Data as External Controls: Practical Experience from Notable Marketing Applications of New Therapies.

INTRODUCTION: Real-world data (RWD) can contextualize findings from single-arm trials when randomized comparative trials are unethical or unfeasible. Findings from single-arm trials alone are difficult to interpret and a comparison, when feasible and meaningful, to patient-level information from RWD facilitates the evaluation. As such, there have been several recent regulatory applications including RWD or other external data to support the product's efficacy and safety. This paper summarizes some lessons learned from such contextualization from 20 notable new drug or biologic licensing applications in oncology and rare diseases. METHODS: This review focuses on 20 notable new drug or biologic licensing applications that included patient-level RWD or other external data for contextualization of trial results. Publicly available regulatory documents including clinical and statistical reviews, advisory committee briefing materials and minutes, and approved product labeling were retrieved for each application. The authors conducted independent assessments of these documents focusing on the regulatory evaluation, in each case. Three examples are presented in detail to illustrate the salient issues and themes identified across applications. RESULTS: Regulatory decisions were strongly influenced by the quality and usability of the RWD. Comparability of cohort attributes such as endpoints, populations, follow-up, index and censoring criteria, as well as data completeness and accuracy of key variables appeared to be essential to ensure the quality and relevance of the RWD. Given adequate sample size of the clinical trials or external control, the use of appropriate analytic methods to properly account for confounding, such as regression or matching, and pre-specification of these methods while blinded to patient outcomes seemed good strategies to address baseline differences. DISCUSSION: Contextualizing single-arm trials with patient-level RWD appears to be an advance in regulatory science; however, challenges remain. Statisticians and epidemiologists have long focused on analytical methods for comparative effectiveness but hurdles in use of RWD have often occurred upstream of the analyses. More specifically, we noted hurdles in evaluating data quality, justifying cohort selection or initiation of follow-up, and demonstrating comparability of cohorts and endpoints.

Design and Analysis of Studies Based on Hierarchical Composite Endpoints: Insights from the DARE-19 Trial.

BACKGROUND/AIM: DARE-19 (NCT04350593) was a randomized trial studying the effects of dapagliflozin, an SGLT2 inhibitor, in hospitalized patients with COVID-19 pneumonia and cardiometabolic risk factors. The conduct of DARE-19 offered the opportunity to define an innovative and clinically meaningful endpoint in a new disease that would best reflect the known profile of dapagliflozin, accompanied by the statistical challenges of analysis and interpretation of such a novel endpoint. METHODS: Hierarchical composite endpoints (HCEs) are based on clinical outcomes which, unlike traditional composite endpoints incorporate ranking of components according to clinical importance. Design of an HCE requires the clinical considerations specific to the therapeutic area under study and the mechanism of action of the investigational treatment. Statistical aspects for the clinical endpoints include the proper definition of the estimand as suggested by ICH E9(R1) for the precise specification of the treatment effect measured by an HCE. RESULTS: We describe the estimand of the DARE-19 trial, where an HCE was constructed to capture the treatment effect of dapagliflozin in hospitalized patients with COVID-19, and was analyzed using a win odds. Practical aspects of designing new studies based on an HCE are described. These include sample size, power, and minimal detectable effect calculations for an HCE based on the win odds analysis, as well as handling of missing data and the clinical interpretability of the win odds in relation to the estimand. CONCLUSIONS: HCEs are flexible endpoints that can be adapted for use in different therapeutic areas, with win odds as the analysis method. DARE-19 is an example of a COVID-19 trial with an HCE as one of the primary endpoints for estimating a clinically interpretable treatment effect in the COVID-19 setting.

Dapagliflozin and Kidney Outcomes in Hospitalized Patients with COVID-19 Infection: An Analysis of the DARE-19 Randomized Controlled Trial.

BACKGROUND AND OBJECTIVES: Patients who were hospitalized with coronavirus disease 2019 (COVID-19) infection are at high risk of AKI and KRT, especially in the presence of CKD. The Dapagliflozin in Respiratory Failure in Patients with COVID-19 (DARE-19) trial showed that in patients hospitalized with COVID-19, treatment with dapagliflozin versus placebo resulted in numerically fewer participants who experienced organ failure or death, although these differences were not statistically significant. We performed a secondary analysis of the DARE-19 trial to determine the efficacy and safety of dapagliflozin on kidney outcomes in the overall population and in prespecified subgroups of participants defined by baseline eGFR. DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS: The DARE-19 trial randomized 1250 patients who were hospitalized (231 [18%] had eGFR <60 ml/min per 1.73 m2) with COVID-19 and cardiometabolic risk factors to dapagliflozin or placebo. Dual primary outcomes (time to new or worsened organ dysfunction or death, and a hierarchical composite end point of recovery [change in clinical status by day 30]), and the key secondary kidney outcome (composite of AKI, KRT, or death), and safety were assessed in participants with baseline eGFR <60 and ≥60 ml/min per 1.73 m2. RESULTS: The effect of dapagliflozin versus placebo on the primary prevention outcome (hazard ratio, 0.80; 95% confidence interval, 0.58 to 1.10), primary recovery outcome (win ratio, 1.09; 95% confidence interval, 0.97 to 1.22), and the composite kidney outcome (hazard ratio, 0.74; 95% confidence interval, 0.50 to 1.07) were consistent across eGFR subgroups (P for interaction: 0.98, 0.67, and 0.44, respectively). The effects of dapagliflozin on AKI were also similar in participants with eGFR <60 ml/min per 1.73 m2 (hazard ratio, 0.71; 95% confidence interval, 0.29 to 1.77) and ≥60 ml/min per 1.73 m2 (hazard ratio, 0.69; 95% confidence interval, 0.37 to 1.29). Dapagliflozin was well tolerated in participants with eGFR <60 and ≥60 ml/min per 1.73 m2. CONCLUSIONS: The effects of dapagliflozin on primary and secondary outcomes in hospitalized participants with COVID-19 were consistent in those with eGFR below/above 60 ml/min per 1.73 m2. Dapagliflozin was well tolerated and did not increase the risk of AKI in participants with eGFR below or above 60 ml/min per 1.73 m2.

Power and sample size calculation for the win odds test: application to an ordinal endpoint in COVID-19 trials.

The win odds is a distribution-free method of comparing locations of distributions of two independent random variables. Introduced as a method for analyzing hierarchical composite endpoints, it is well suited to be used in the analysis of ordinal scale endpoints in COVID-19 clinical trials. For a single outcome, we provide power and sample size calculation formulas for the win odds test. We also provide an implementation of the win odds analysis method for a single ordinal outcome in a commonly used statistical software to make the win odds analysis fully reproducible.

Dapagliflozin in patients with cardiometabolic risk factors hospitalised with COVID-19 (DARE-19): a randomised, double-blind, placebo-controlled, phase 3 trial.

BACKGROUND: COVID-19 can lead to multiorgan failure. Dapagliflozin, a SGLT2 inhibitor, has significant protective benefits for the heart and kidney. We aimed to see whether this agent might provide organ protection in patients with COVID-19 by affecting processes dysregulated during acute illness. METHODS: DARE-19 was a randomised, double-blind, placebo-controlled trial of patients hospitalised with COVID-19 and with at least one cardiometabolic risk factor (ie, hypertension, type 2 diabetes, atherosclerotic cardiovascular disease, heart failure, and chronic kidney disease). Patients critically ill at screening were excluded. Patients were randomly assigned 1:1 to dapagliflozin (10 mg daily orally) or matched placebo for 30 days. Dual primary outcomes were assessed in the intention-to-treat population: the outcome of prevention (time to new or worsened organ dysfunction or death), and the hierarchial composite outcome of recovery (change in clinical status by day 30). Safety outcomes, in patients who received at least one study medication dose, included serious adverse events, adverse events leading to discontinuation, and adverse events of interest. This study is registered with ClinicalTrials.gov, NCT04350593. FINDINGS: Between April 22, 2020 and Jan 1, 2021, 1250 patients were randomly assigned with 625 in each group. The primary composite outcome of prevention showed organ dysfunction or death occurred in 70 patients (11·2%) in the dapagliflozin group, and 86 (13·8%) in the placebo group (hazard ratio [HR] 0·80, 95% CI 0·58-1·10; p=0·17). For the primary outcome of recovery, 547 patients (87·5%) in the dapagliflozin group and 532 (85·1%) in the placebo group showed clinical status improvement, although this was not statistically significant (win ratio 1·09, 95% CI 0·97-1·22; p=0·14). There were 41 deaths (6·6%) in the dapagliflozin group, and 54 (8·6%) in the placebo group (HR 0·77, 95% CI 0·52-1·16). Serious adverse events were reported in 65 (10·6%) of 613 patients treated with dapagliflozin and in 82 (13·3%) of 616 patients given the placebo. INTERPRETATION: In patients with cardiometabolic risk factors who were hospitalised with COVID-19, treatment with dapagliflozin did not result in a statistically significant risk reduction in organ dysfunction or death, or improvement in clinical recovery, but was well tolerated. FUNDING: AstraZeneca.

Effects of dapagliflozin on prevention of major clinical events and recovery in patients with respiratory failure because of COVID-19: Design and rationale for the DARE-19 study.

AIMS: Coronavirus disease 2019 (COVID-19) is caused by a novel severe acute respiratory syndrome coronavirus 2. It can lead to multiorgan failure, including respiratory and cardiovascular decompensation, and kidney injury, with significant associated morbidity and mortality, particularly in patients with underlying metabolic, cardiovascular, respiratory or kidney disease. Dapagliflozin, a sodium-glucose cotransporter-2 inhibitor, has shown significant cardio- and renoprotective benefits in patients with type 2 diabetes (with and without atherosclerotic cardiovascular disease), heart failure and chronic kidney disease, and may provide similar organ protection in high-risk patients with COVID-19. MATERIALS AND METHODS: DARE-19 (NCT04350593) is an investigator-initiated, collaborative, international, multicentre, randomized, double-blind, placebo-controlled study testing the dual hypotheses that dapagliflozin can reduce the incidence of cardiovascular, kidney and/or respiratory complications or all-cause mortality, or improve clinical recovery, in adult patients hospitalized with COVID-19 but not critically ill on admission. Eligible patients will have ≥1 cardiometabolic risk factor for COVID-19 complications. Patients will be randomized 1:1 to dapagliflozin 10 mg or placebo. Primary efficacy endpoints are time to development of new or worsened organ dysfunction during index hospitalization, or all-cause mortality, and the hierarchical composite endpoint of change in clinical status through day 30 of treatment. Safety of dapagliflozin in individuals with COVID-19 will be assessed. CONCLUSIONS: DARE-19 will evaluate whether dapagliflozin can prevent COVID-19-related complications and all-cause mortality, or improve clinical recovery, and assess the safety profile of dapagliflozin in this patient population. Currently, DARE-19 is the first large randomized controlled trial investigating use of sodium-glucose cotransporter 2 inhibitors in patients with COVID-19.

Adjusted win ratio with stratification: Calculation methods and interpretation.

The win ratio is a general method of comparing locations of distributions of two independent, ordinal random variables, and it can be estimated without distributional assumptions. In this paper we provide a unified theory of win ratio estimation in the presence of stratification and adjustment by a numeric variable. Building step by step on the estimate of the crude win ratio we compare corresponding tests with well known non-parametric tests of group difference (Wilcoxon rank-sum test, Fligner-Policello test, van Elteren test, test based on the regression on ranks, and the rank analysis of covariance test). We show that the win ratio gives an interpretable treatment effect measure with corresponding test to detect treatment effect difference under minimal assumptions.

A Bayesian hierarchical model for the estimation of two incomplete surveillance data sets.

A model-based approach to analyze two incomplete disease surveillance datasets is described. Such data typically consist of case counts, each originating from a specific geographical area. A Bayesian hierarchical model is proposed for estimating the total number of cases with disease while simultaneously adjusting for spatial variation. This approach explicitly accounts for model uncertainty and can make use of covariates.The method is applied to two surveillance datasets maintained by the Centers for Disease Control and Prevention on Rocky Mountain spotted fever (RMSF). An inference is drawn using Markov Chain Monte Carlo simulation techniques in a fully Bayesian framework. The central feature of the model is the ability to calculate and estimate the total number of cases and disease incidence for geographical regions where RMSF is endemic.The information generated by this model could significantly reduce the public health impact of RMSF and other vector-borne zoonoses, as well as other infectious or chronic diseases, by improving knowledge of the spatial distribution of disease risk of public health officials and medical practitioners. More accurate information on populations at high risk would focus attention and resources on specific areas, thereby reducing the morbidity and mortality caused by some of the preventable and treatable diseases.

Impact of valve surgery on 6-month mortality in adults with complicated, left-sided native valve endocarditis: a propensity analysis.

CONTEXT: Complicated, left-sided native valve endocarditis causes significant morbidity and mortality in adults. The presumed benefits of valve surgery remain unproven due to lack of randomized controlled trials. OBJECTIVE: To determine whether valve surgery is associated with reduced mortality in adults with complicated, left-sided native valve endocarditis. DESIGN AND SETTING: Retrospective, observational cohort study conducted from January 1990 to January 2000 at 7 Connecticut hospitals. Propensity analyses were used to control for bias in treatment assignment and prognostic imbalances. PATIENTS: Of the 513 adults with complicated, left-sided native valve endocarditis, 230 (45%) underwent valve surgery and 283 (55%) received medical therapy alone. MAIN OUTCOME MEASURE: All-cause mortality at 6 months after baseline. RESULTS: In the 6-month period after baseline, 131 patients (26%) died. In unadjusted analyses, valve surgery was associated with reduced mortality (16% vs 33%; hazard ratio [HR], 0.43; 95% confidence interval [CI], 0.29-0.63; P<.001). After adjustment for baseline variables associated with mortality (including hospital site, comorbidity, congestive heart failure, microbial etiology, immunocompromised state, abnormal mental status, and refractory infection), valve surgery remained associated with reduced mortality (adjusted HR, 0.35; 95% CI, 0.23-0.54; P<.02). In further analyses of 218 patients matched by propensity scores, valve surgery remained associated with reduced mortality (15% vs 28%; HR, 0.45; 95% CI, 0.23-0.86; P =.01). After additional adjustment for variables that contribute to heterogeneity and confounding within the propensity-matched group, surgical therapy remained significantly associated with a lower mortality (HR, 0.40; 95% CI, 0.18-0.91; P =.03). In this propensity-matched group, patients with moderate to severe congestive heart failure showed the greatest reduction in mortality with valve surgery (14% vs 51%; HR, 0.22; 95% CI, 0.09-0.53; P =.001). CONCLUSIONS: Valve surgery for patients with complicated, left-sided native valve endocarditis was independently associated with reduced 6-month mortality after adjustment for both baseline variables associated with the propensity to undergo valve surgery and baseline variables associated with mortality. The reduced mortality was particularly evident among patients with moderate to severe congestive heart failure.

Complicated left-sided native valve endocarditis in adults: risk classification for mortality.

CONTEXT: Complicated left-sided native valve endocarditis causes significant morbidity and mortality in adults. Lack of valid data regarding estimation of prognosis makes management of this condition difficult. OBJECTIVE: To derive and externally validate a prognostic classification system for adults with complicated left-sided native valve endocarditis. DESIGN, SETTING, AND PATIENTS: Retrospective observational cohort study conducted from January 1990 to January 2000 at 7 Connecticut hospitals among 513 patients older than 16 years who experienced complicated left-sided native valve endocarditis and who were divided into derivation (n = 259) and validation (n = 254) cohorts. MAIN OUTCOME MEASURE: All-cause mortality at 6 months after baseline. RESULTS: In the derivation and validation cohorts, the 6-month mortality rates were 25% and 26%, respectively. Five baseline features were independently associated with 6-month mortality (comorbidity [P =.03], abnormal mental status [P =.02], moderate to severe congestive heart failure [P =.01], bacterial etiology other than viridans streptococci [P<.001 except Staphylococcus aureus, P =.004], and medical therapy without valve surgery [P =.002]) and were used to create a prognostic classification system. In the derivation cohort, patients were classified into 4 groups with increasing risk for 6-month mortality: 5%, 15%, 31%, and 59% (P<.001). In the validation cohort, a similar risk among the 4 groups was observed: 7%, 19%, 32%, and 69% (P<.001). CONCLUSIONS: Adults with complicated left-sided native valve endocarditis can be accurately risk stratified using baseline features into 4 groups of prognostic severity. This prognostic classification system might be useful for facilitating management decisions.

Disability and treatment patterns of multiple sclerosis patients in United States: a comparison of veterans and nonveterans.

The Veterans Health Administration (VHA) is the largest integrated healthcare system in the world and provides care to approximately 20,000 multiple sclerosis (MS) patients. Here, we report that these MS patients are disproportionately more likely to be older, male, unemployed, and disabled with lower levels of education and financial resources when compared to veterans not receiving care within the VHA or to nonveteran MS patients. When comparing the VHA MS patients to a cohort of nonveteran MS patients matched for age, sex, and disability, we found that veterans receiving care within the VHA were equally likely to have received care from a neurologist and more likely to have received care from rehabilitation specialists and primary care physicians than nonveterans. Similarly, veterans in the VHA were more likely to receive therapy with certain symptomatic medications but were less likely to be treated with disease-modifying agents for MS (DMAMS) than nonveterans. When treated with DMAMS, they are more likely to be treated with Avonex and significantly less likely to receive treatment with Copaxone or Novantrone.