Risk factors and prediction of hypoglycaemia using the Hypo-RESOLVE cohort: a secondary analysis of pooled data from insulin clinical trials.

AIMS/HYPOTHESIS: The objective of the Hypoglycaemia REdefining SOLutions for better liVES (Hypo-RESOLVE) project is to use a dataset of pooled clinical trials across pharmaceutical and device companies in people with type 1 or type 2 diabetes to examine factors associated with incident hypoglycaemia events and to quantify the prediction of these events. METHODS: Data from 90 trials with 46,254 participants were pooled. Analyses were done for type 1 and type 2 diabetes separately. Poisson mixed models, adjusted for age, sex, diabetes duration and trial identifier were fitted to assess the association of clinical variables with hypoglycaemia event counts. Tree-based gradient-boosting algorithms (XGBoost) were fitted using training data and their predictive performance in terms of area under the receiver operating characteristic curve (AUC) evaluated on test data. Baseline models including age, sex and diabetes duration were compared with models that further included a score of hypoglycaemia in the first 6 weeks from study entry, and full models that included further clinical variables. The relative predictive importance of each covariate was assessed using XGBoost's importance procedure. Prediction across the entire trial duration for each trial (mean of 34.8 weeks for type 1 diabetes and 25.3 weeks for type 2 diabetes) was assessed. RESULTS: For both type 1 and type 2 diabetes, variables associated with more frequent hypoglycaemia included female sex, white ethnicity, longer diabetes duration, treatment with human as opposed to analogue-only insulin, higher glucose variability, higher score for hypoglycaemia across the 6 week baseline period, lower BP, lower lipid levels and treatment with psychoactive drugs. Prediction of any hypoglycaemia event of any severity was greater than prediction of hypoglycaemia requiring assistance (level 3 hypoglycaemia), for which events were sparser. For prediction of level 1 or worse hypoglycaemia during the whole follow-up period, the AUC was 0.835 (95% CI 0.826, 0.844) in type 1 diabetes and 0.840 (95% CI 0.831, 0.848) in type 2 diabetes. For level 3 hypoglycaemia, the AUC was lower at 0.689 (95% CI 0.667, 0.712) for type 1 diabetes and 0.705 (95% CI 0.662, 0.748) for type 2 diabetes. Compared with the baseline models, almost all the improvement in prediction could be captured by the individual's hypoglycaemia history, glucose variability and blood glucose over a 6 week baseline period. CONCLUSIONS/INTERPRETATION: Although hypoglycaemia rates show large variation according to sociodemographic and clinical characteristics and treatment history, looking at a 6 week period of hypoglycaemia events and glucose measurements predicts future hypoglycaemia risk.

Estimating risk of consequences following hypoglycaemia exposure using the Hypo-RESOLVE cohort: a secondary analysis of pooled data from insulin clinical trials.

AIMS/HYPOTHESIS: Whether hypoglycaemia increases the risk of other adverse outcomes in diabetes remains controversial, especially for hypoglycaemia episodes not requiring assistance from another person. An objective of the Hypoglycaemia REdefining SOLutions for better liVEs (Hypo-RESOLVE) project was to create and use a dataset of pooled clinical trials in people with type 1 or type 2 diabetes to examine the association of exposure to all hypoglycaemia episodes across the range of severity with incident event outcomes: death, CVD, neuropathy, kidney disease, retinal disorders and depression. We also examined the change in continuous outcomes that occurred following a hypoglycaemia episode: change in eGFR, HbA1c, blood glucose, blood glucose variability and weight. METHODS: Data from 84 trials with 39,373 participants were pooled. For event outcomes, time-updated Cox regression models adjusted for age, sex, diabetes duration and HbA1c were fitted to assess association between: (1) outcome and cumulative exposure to hypoglycaemia episodes; and (2) outcomes where an acute effect might be expected (i.e. death, acute CVD, retinal disorders) and any hypoglycaemia exposure within the last 10 days. Exposures to any hypoglycaemia episode and to episodes of given severity (levels 1, 2 and 3) were examined. Further adjustment was then made for a wider set of potential confounders. The within-person change in continuous outcomes was also summarised (median of 40.4 weeks for type 1 diabetes and 26 weeks for type 2 diabetes). Analyses were conducted separately by type of diabetes. RESULTS: The maximally adjusted association analysis for type 1 diabetes found that cumulative exposure to hypoglycaemia episodes of any level was associated with higher risks of neuropathy, kidney disease, retinal disorders and depression, with risk ratios ranging from 1.55 (p=0.002) to 2.81 (p=0.002). Associations of a similar direction were found when level 1 episodes were examined separately but were significant for depression only. For type 2 diabetes cumulative exposure to hypoglycaemia episodes of any level was associated with higher risks of death, acute CVD, kidney disease, retinal disorders and depression, with risk ratios ranging from 2.35 (p<0.0001) to 3.00 (p<0.0001). These associations remained significant when level 1 episodes were examined separately. There was evidence of an association between hypoglycaemia episodes of any kind in the previous 10 days and death, acute CVD and retinal disorders in both type 1 and type 2 diabetes, with rate ratios ranging from 1.32 (p=0.017) to 2.68 (p<0.0001). These associations varied in magnitude and significance when examined separately by hypoglycaemia level. Within the range of hypoglycaemia defined by levels 1, 2 and 3, we could not find any evidence of a threshold at which risk of these consequences suddenly became pronounced. CONCLUSIONS/INTERPRETATION: These data are consistent with hypoglycaemia being associated with an increased risk of adverse events across several body systems in diabetes. These associations are not confined to severe hypoglycaemia requiring assistance.

A unified Bayesian framework for bias adjustment in multiple comparisons from clinical trials.

In clinical trials, multiple comparisons arising from various treatments/doses, subgroups, or endpoints are common. Typically, trial teams focus on the comparison showing the largest observed treatment effect, often involving a specific treatment pair and endpoint within a subgroup. These findings frequently lead to follow-up pivotal studies, many of which do not confirm the initial positive results. Selection bias occurs when the most promising treatment, subgroup, or endpoint is chosen for further development, potentially skewing subsequent investigations. Such bias can be defined as the deviation in the observed treatment effects from the underlying truth. In this article, we propose a general and unified Bayesian framework to address selection bias in clinical trials with multiple comparisons. Our approach does not require a priori specification of a parametric distribution for the prior, offering a more flexible and generalized solution. The proposed method facilitates a more accurate interpretation of clinical trial results by adjusting for such selection bias. Through simulation studies, we compared several methods and demonstrated their superior performance over the normal shrinkage estimator. We recommended the use of Bayesian Model Averaging estimator averaging over Gaussian Mixture Models as the prior distribution based on its performance and flexibility. We applied the method to a multicenter, randomized, double-blind, placebo-controlled study investigating the cardiovascular effects of dulaglutide.

CITIES: Clinical trials with intercurrent events simulator.

Although clinical trials are often designed with randomization and well-controlled protocols, complications will inevitably arise in the presence of intercurrent events (ICEs) such as treatment discontinuation. These can lead to missing outcome data and possibly confounding causal inference when the missingness is a function of a latent stratification of patients defined by intermediate outcomes. The pharmaceutical industry has been focused on developing new methods that can yield pertinent causal inferences in trials with ICEs. However, it is difficult to compare the properties of different methods developed in this endeavor as real-life clinical trial data cannot be easily shared to provide benchmark data sets. Furthermore, different methods consider distinct assumptions for the underlying data-generating mechanisms, and simulation studies often are customized to specific situations or methods. We develop a novel, general simulation model and corresponding Shiny application in R for clinical trials with ICEs, aptly named the Clinical Trials with Intercurrent Events Simulator (CITIES). It is formulated under the Rubin Causal Model where the considered treatment effects account for ICEs in clinical trials with repeated measures. CITIES facilitates the effective generation of data that resemble real-life clinical trials with respect to their reported summary statistics, without requiring the use of the original trial data. We illustrate the utility of CITIES via two case studies involving real-life clinical trials that demonstrate how CITIES provides a comprehensive tool for practitioners in the pharmaceutical industry to compare methods for the analysis of clinical trials with ICEs on identical, benchmark settings that resemble real-life trials.

Using principal stratification in analysis of clinical trials.

The ICH E9(R1) addendum (2019) proposed principal stratification (PS) as one of five strategies for dealing with intercurrent events. Therefore, understanding the strengths, limitations, and assumptions of PS is important for the broad community of clinical trialists. Many approaches have been developed under the general framework of PS in different areas of research, including experimental and observational studies. These diverse applications have utilized a diverse set of tools and assumptions. Thus, need exists to present these approaches in a unifying manner. The goal of this tutorial is threefold. First, we provide a coherent and unifying description of PS. Second, we emphasize that estimation of effects within PS relies on strong assumptions and we thoroughly examine the consequences of these assumptions to understand in which situations certain assumptions are reasonable. Finally, we provide an overview of a variety of key methods for PS analysis and use a real clinical trial example to illustrate them. Examples of code for implementation of some of these approaches are given in Supplemental Materials.

Return-to-baseline multiple imputation for missing values in clinical trials.

Return-to-baseline is an important method to impute missing values or unobserved potential outcomes when certain hypothetical strategies are used to handle intercurrent events in clinical trials. Current return-to-baseline approaches seen in literature and in practice inflate the variability of the "complete" dataset after imputation and lead to biased mean estimators when the probability of missingness depends on the observed baseline and/or postbaseline intermediate outcomes. In this article, we first provide a set of criteria a return-to-baseline imputation method should satisfy. Under this framework, we propose a novel return-to-baseline imputation method. Simulations show the completed data after the new imputation approach have the proper distribution, and the estimators based on the new imputation method outperform the traditional method in terms of both bias and variance, when missingness depends on the observed values. The new method can be implemented easily with the existing multiple imputation procedures in commonly used statistical packages.

Analysis of an incomplete binary outcome dichotomized from an underlying continuous variable in clinical trials.

In many clinical trials, outcomes of interest are binary-valued. It is not uncommon that a binary-valued outcome is dichotomized from a continuous outcome at a threshold of clinical interest. To analyze such data, common approaches include (a) fitting a generalized linear mixed model (GLMM) to the dichotomized longitudinal binary outcome; and (b) the multiple imputation (MI) based method: imputing missing values in the continuous outcome, dichotomizing it into a binary outcome, and then fitting a generalized linear model to the "complete" data. We conducted comprehensive simulation studies to compare the performance of the GLMM versus the MI-based method for estimating the risk difference and the logarithm of odds ratio between two treatment arms at the end of study. In those simulation studies, we considered a range of multivariate distribution options for the continuous outcome (including a multivariate normal distribution, a multivariate t-distribution, a multivariate log-normal distribution, and the empirical distribution from a real clinical trial data) to evaluate the robustness of the estimators to various data-generating models. Simulation results demonstrate that both methods work well under those considered distribution options, but the MI-based method is more efficient with smaller mean squared errors compared to the GLMM. We further applied both the GLMM and the MI-based method to 29 phase 3 diabetes clinical trials, and found that the MI-based method generally led to smaller variance estimates compared to the GLMM.

Estimating the treatment effect for adherers using multiple imputation.

Randomized controlled trials are considered the gold standard to evaluate the treatment effect (estimand) for efficacy and safety. According to the recent International Council on Harmonization (ICH)-E9 addendum (R1), intercurrent events (ICEs) need to be considered when defining an estimand, and principal stratum is one of the five strategies to handle ICEs. Qu et al. (2020, Statistics in Biopharmaceutical Research 12:1-18) proposed estimators for the adherer average causal effect (AdACE) for estimating the treatment difference for those who adhere to one or both treatments based on the causal-inference framework, and demonstrated the consistency of those estimators; however, this method requires complex custom programming related to high-dimensional numeric integrations. In this article, we implemented the AdACE estimators using multiple imputation (MI) and constructed confidence intervals (CIs) through bootstrapping. A simulation study showed that the MI-based estimators provided consistent estimators with the nominal coverage probabilities of CIs for the treatment difference for the adherent populations of interest. As an illustrative example, the new method was applied to data from a real clinical trial comparing two types of basal insulin for patients with type 1 diabetes.

A unified approach for evaluating the prediction of treatment effect across different types of endpoints.

Phase 2 and 3 development failure is one of the key factors for high drug development cost. Robust prediction of a candidate drug's efficacy and safety profile could potentially improve the success rate of the drug development. Therefore, systematic evaluation of the prediction is important for learning and continuous improvement of the prediction. In this article, we proposed a set of unified criteria that allow to evaluate the predictions across different endpoints, indications and development stages: standardized bias (SB), standardized mean squared errors (SMSE), and credibility of prediction. We applied the SB and SMSE to the predicted treatment effects for 54 comparisons in 5 compounds in immunology and diabetes.

Analysis of recurrent hypoglycemic events.

Hypoglycemia is a major safety concern for diabetic patients. Hypoglycemic events can be modeled based on time to recurrent events or count data. In this article, we evaluated a gamma frailty model with variance estimated by the inverse of observed Fisher information matrix, a gamma frailty model with the sandwich variance estimator, and a piecewise negative binomial regression model. Simulations showed that the sandwich variance estimator performed better when the frailty model is mis-specified, and the piecewise negative binomial regression sometimes fails to converge. All three methods were applied to a dataset from a clinical trial evaluating insulin treatments.

Implementation of tripartite estimands using adherence causal estimators under the causal inference framework.

Intercurrent events (ICEs) and missing values are inevitable in clinical trials of any size and duration, making it difficult to assess the treatment effect for all patients in randomized clinical trials. Defining the appropriate estimand that is relevant to the clinical research question is the first step in analyzing data. The tripartite estimands, which evaluate the treatment differences in the proportion of patients with ICEs due to adverse events, the proportion of patients with ICEs due to lack of efficacy, and the primary efficacy outcome for those who can adhere to study treatment under the causal inference framework, are of interest to many stakeholders in understanding the totality of treatment effects. In this manuscript, we discuss the details of how to estimate tripartite estimands based on a causal inference framework and how to interpret tripartite estimates through a phase 3 clinical study evaluating a basal insulin treatment for patients with type 1 diabetes.

Defining estimands using a mix of strategies to handle intercurrent events in clinical trials.

Randomized controlled trials (RCTs) are the gold standard for evaluation of the efficacy and safety of investigational interventions. If every patient in an RCT were to adhere to the randomized treatment, one could simply analyze the complete data to infer the treatment effect. However, intercurrent events (ICEs) including the use of concomitant medication for unsatisfactory efficacy, treatment discontinuation due to adverse events, or lack of efficacy may lead to interventions that deviate from the original treatment assignment. Therefore, defining the appropriate estimand (the appropriate parameter to be estimated) based on the primary objective of the study is critical prior to determining the statistical analysis method and analyzing the data. The International Council for Harmonisation (ICH) E9 (R1), adopted on November 20, 2019, provided five strategies to define the estimand: treatment policy, hypothetical, composite variable, while on treatment, and principal stratum. In this article, we propose an estimand using a mix of strategies in handling ICEs. This estimand is an average of the "null" treatment difference for those with ICEs potentially related to safety and the treatment difference for the other patients if they would complete the assigned treatments. Two examples from clinical trials evaluating antidiabetes treatments are provided to illustrate the estimation of this proposed estimand and to compare it with the estimates for estimands using hypothetical and treatment policy strategies in handling ICEs.

Proportion of treatment effect mediated by surrogate endpoints.

One of the central aims in randomized clinical trials is to find well-validated surrogate endpoints to reduce the sample size and/or duration of trials. Clinical researchers and practitioners have proposed various surrogacy measures for assessing candidate surrogate endpoints. However, most existing surrogacy measures have the following shortcomings: (i) they often fall outside the range [0,1], (ii) they are imprecisely estimated, and (iii) they ignore the interaction associations between a treatment and candidate surrogate endpoints in the evaluation of the surrogacy level. To overcome these difficulties, we propose a new surrogacy measure, the proportion of treatment effect mediated by candidate surrogate endpoints (PMS), based on the decomposition of the treatment effect into direct, indirect, and interaction associations mediated by candidate surrogate endpoints. In addition, we validate the advantages of PMS through Monte Carlo simulations and the application to empirical data from ORIENT (the Olmesartan Reducing Incidence of Endstage Renal Disease in Diabetic Nephropathy Trial).

Understanding and adjusting for the selection bias from a proof-of-concept study to a more confirmatory study.

It has long been noticed that the efficacy observed in small early phase studies is generally better than that observed in later larger studies. Historically, the inflation of the efficacy results from early proof-of-concept studies is either ignored, or adjusted empirically using a frequentist or Bayesian approach. In this article, we systematically explained the underlying reason for the inflation of efficacy results in small early phase studies from the perspectives of measurement error models and selection bias. A systematic method was built to adjust the early phase study results from both frequentist and Bayesian perspectives. A hierarchical model was proposed to estimate the distribution of the efficacy for a portfolio of compounds, which can serve as the prior distribution for the Bayesian approach. We showed through theory that the systematic adjustment provides an unbiased estimator for the true mean efficacy for a portfolio of compounds. The adjustment was applied to paired data for the efficacy in early small and later larger studies for a set of compounds in diabetes and immunology. After the adjustment, the bias in the early phase small studies seems to be diminished.

Estimation of group means using Bayesian generalized linear mixed models.

Generalized linear mixed models (GLMM) are commonly used to model the treatment effect over time while controlling for important clinical covariates. Standard software procedures often provide estimates of the outcome based on the mean of the covariates; however, these estimates will be biased for the true group means in the GLMM. Implementing GLMM in the frequentist framework can lead to issues of convergence. A simulation study demonstrating the use of fully Bayesian GLMM for providing unbiased estimates of group means is shown. These models are very straightforward to implement and can be used for a broad variety of outcomes (eg, binary, categorical, and count data) that arise in clinical trials. We demonstrate the proposed method on a data set from a clinical trial in diabetes.

Estimation of group means in generalized linear mixed models.

In this study, we investigate the concept of the mean response for a treatment group mean as well as its estimation and prediction for generalized linear models with a subject-wise random effect. Generalized linear models are commonly used to analyze categorical data. The model-based mean for a treatment group usually estimates the response at the mean covariate. However, the mean response for the treatment group for studied population is at least equally important in the context of clinical trials. New methods were proposed to estimate such a mean response in generalized linear models; however, this has only been done when there are no random effects in the model. We suggest that, in a generalized linear mixed model (GLMM), there are at least two possible definitions of a treatment group mean response that can serve as estimation/prediction targets. The estimation of these treatment group means is important for healthcare professionals to be able to understand the absolute benefit vs risk. For both of these treatment group means, we propose a new set of methods that suggests how to estimate/predict both of them in a GLMMs with a univariate subject-wise random effect. Our methods also suggest an easy way of constructing corresponding confidence and prediction intervals for both possible treatment group means. Simulations show that proposed confidence and prediction intervals provide correct empirical coverage probability under most circumstances. Proposed methods have also been applied to analyze hypoglycemia data from diabetes clinical trials.

Can a multiple ascending dose study serve as an informative proof-of-concept study?

Drug development is a long, complex, and costly process. The majority of the cost arises from Phase 2 and Phase 3 clinical development. Reducing Phase 2 and Phase 3 failure rates would greatly reduce the average drug development cost. Obtaining more informative data on a candidate drug's efficacy and safety prior to moving to Phase 2 will improve Phase 2 success and, hence, reduce the overall development cost. While, typically, multiple ascending dose (MAD) study focuses on safety, this article proposes a model-based MAD design that not only can provide the desired safety information but also can provide informative efficacy data for certain endpoints in certain therapeutic areas where the parametric models for the longitudinal dose response and the impact of dose titration on a response variable are utilized. This type of MAD design allows relatively informative efficacy data available prior to Phase 2 development and, hence, can serve as a proof-of-concept study. This approach may greatly reduce the drug development cycle time without increasing the risk of Phase 2 development.

Modeling the impact of preplanned dose titration on delayed response.

Dose titration becomes more and more common in improving drug tolerability as well as determining individualized treatment doses, thereby maximizing the benefit to patients. Dose titration starting from a lower dose and gradually increasing to a higher dose enables improved tolerability in patients as the human body may gradually adapt to adverse gastrointestinal effects. Current statistical analyses mostly focus on the outcome at the end-of-study follow-up without considering the longitudinal impact of dose titration on the outcome. Better understanding of the dynamic effect of dose titration over time is important in early-phase clinical development as it could allow to model the longitudinal trend and predict the longer term outcome more accurately. We propose a parametric model with two empirical methods of modeling the error terms for a continuous outcome with dose titrations. Simulations show that both approaches of modeling the error terms work well. We applied this method to analyze data from a few clinical studies and achieved satisfactory results.

Extending the classification approach for comparing two active treatment arms to binary and time-to-event outcomes.

For regulatory purposes, in a trial comparing two active treatments, a hypothesis such as noninferiority or superiority must be prespecified even when there is little known about how they compare against each other or when the objective is simply to identify the best. In this article, we extend an alternative classification methodology, the classification approach of Qu et al. (Statistics in Medicine, 30:3488-3495), to compare two active treatments when outcomes are binary and time-to-event variables. This method based on estimation approach instead of hypothesis testing can be useful when little prior information is available on which treatment has better efficacy. The entire decision space is divided into eight distinct possible outcomes based on predefined lower and upper non-inferiority margins, and the conclusion will be drawn according to the location of the confidence interval for relative risk or hazard ratio (or its logarithm transformation). We demonstrate theoretically that this method controls the misclassification rate at the specified level. We also illustrate the method by simulations and using data from a Phase 3 first-line nonsmall cell lung cancer study.