Determining sample size in a personalized randomized controlled (PRACTical) trial.

In clinical settings with no commonly accepted standard-of-care, multiple treatment regimens are potentially useful, but some treatments may not be appropriate for some patients. A personalized randomized controlled trial (PRACTical) design has been proposed for this setting. For a network of treatments, each patient is randomized only among treatments which are appropriate for them. The aim is to produce treatment rankings that can inform clinical decisions about treatment choices for individual patients. Here we propose methods for determining sample size in a PRACTical design, since standard power-based methods are not applicable. We derive a sample size by evaluating information gained from trials of varying sizes. For a binary outcome, we quantify how many adverse outcomes would be prevented by choosing the top-ranked treatment for each patient based on trial results rather than choosing a random treatment from the appropriate personalized randomization list. In simulations, we evaluate three performance measures: mean reduction in adverse outcomes using sample information, proportion of simulated patients for whom the top-ranked treatment performed as well or almost as well as the best appropriate treatment, and proportion of simulated trials in which the top-ranked treatment performed better than a randomly chosen treatment. We apply the methods to a trial evaluating eight different combination antibiotic regimens for neonatal sepsis (NeoSep1), in which a PRACTical design addresses varying patterns of antibiotic choice based on disease characteristics and resistance. Our proposed approach produces results that are more relevant to complex decision making by clinicians and policy makers.

Regression-based multiple treatment effect estimation under covariate-adaptive randomization.

Covariate-adaptive randomization methods are widely used in clinical trials to balance baseline covariates. Recent studies have shown the validity of using regression-based estimators for treatment effects without imposing functional form requirements on the true data generation model. These studies have had limitations in certain scenarios; for example, in the case of multiple treatment groups, these studies did not consider additional covariates or assumed that the allocation ratios were the same across strata. To address these limitations, we develop a stratum-common estimator and a stratum-specific estimator under multiple treatments. We derive the asymptotic behaviors of these estimators and propose consistent nonparametric estimators for asymptotic variances. To determine their efficiency, we compare the estimators with the stratified difference-in-means estimator as the benchmark. We find that the stratum-specific estimator guarantees efficiency gains, regardless of whether the allocation ratios across strata are the same or different. Our conclusions were also validated by simulation studies and a real clinical trial example.

Introducing the Treatment Hierarchy Question in Network Meta-Analysis.

Comparative effectiveness research using network meta-analysis can present a hierarchy of competing treatments, from the most to the least preferable option. However, in published reviews, the research question associated with the hierarchy of multiple interventions is typically not clearly defined. Here we introduce the novel notion of a treatment hierarchy question that describes the criterion for choosing a specific treatment over one or more competing alternatives. For example, stakeholders might ask which treatment is most likely to improve mean survival by at least 2 years, or which treatment is associated with the longest mean survival. We discuss the most commonly used ranking metrics (quantities that compare the estimated treatment-specific effects), how the ranking metrics produce a treatment hierarchy, and the type of treatment hierarchy question that each ranking metric can answer. We show that the ranking metrics encompass the uncertainty in the estimation of the treatment effects in different ways, which results in different treatment hierarchies. When using network meta-analyses that aim to rank treatments, investigators should state the treatment hierarchy question they aim to address and employ the appropriate ranking metric to answer it. Following this new proposal will avoid some controversies that have arisen in comparative effectiveness research.

Bayesian models for aggregate and individual patient data component network meta-analysis.

Network meta-analysis can synthesize evidence from studies comparing multiple treatments for the same disease. Sometimes the treatments of a network are complex interventions, comprising several independent components in different combinations. A component network meta-analysis (CNMA) can be used to analyze such data and can in principle disentangle the individual effect of each component. However, components may interact with each other, either synergistically or antagonistically. Deciding which interactions, if any, to include in a CNMA model may be difficult, especially for large networks with many components. In this article, we present two Bayesian CNMA models that can be used to identify prominent interactions between components. Our models utilize Bayesian variable selection methods, namely the stochastic search variable selection and the Bayesian LASSO, and can benefit from the inclusion of prior information about important interactions. Moreover, we extend these models to combine data from studies providing aggregate information and studies providing individual patient data (IPD). We illustrate our models in practice using three real datasets, from studies in panic disorder, depression, and multiple myeloma. Finally, we describe methods for developing web-applications that can utilize results from an IPD-CNMA, to allow for personalized estimates of relative treatment effects given a patient's characteristics.

Multiple imputation procedures for estimating causal effects with multiple treatments with application to the comparison of healthcare providers.

Choosing between multiple healthcare providers requires us to simultaneously compare the expected outcomes under each provider. This comparison is complex because the composition of patients treated by each provider may differ. Similar issues arise when simultaneously comparing the adverse effects of interventions using non-randomized data. To simultaneously estimate the effects of multiple providers/interventions we propose procedures that explicitly impute the set of potential outcomes for each subject. The procedures are based on different specifications of the generalized additive models (GAM) and the Bayesian additive regression trees (BART). We compare the performance of the proposed procedures to previously proposed matching and weighting procedures using an extensive simulation study for continuous outcomes. Our simulations show that when the distributions of the covariates across treatment groups have adequate overlap, the multiple imputation procedures based on separate BART or GAM models in each treatment group are generally superior to weighting based methods and have similar and sometimes better performance than matching on the logit of the generalized propensity score. Another advantage of these multiple imputation procedures is the ability to provide point and interval estimates to a wide range of causal effect estimands. We apply the proposed procedures to comparing multiple nursing homes in Massachusetts for readmission outcomes. The proposed approach can be applied to other causal effects applications with multiple treatments.

Utilizing stratified generalized propensity score matching to approximate blocked randomized designs with multiple treatment levels.

Conducting causal inference in settings with more than one treatment level can be challenging. Classical methods, such as propensity score matching (PSM), are restricted to only a binary treatment. To extend propensity score methods beyond a binary treatment, generalized propensity score methods have been proposed, with generalized propensity score matching (GPSM) standing as the multi-level treatment analog to PSM. One drawback of GPSM is it is only capable of emulating a completely randomized trial (CRT) design and not the more efficient blocked randomized trial design. Motivated by the desire to emulate the more efficient design, we expand on GPSM estimating literature and develop a new estimator incorporating relevant stratifying variables into the GPSM framework. We examine the variance estimation methods available for GPSM and demonstrate how to extend the estimator to one where stratifying variables are included. While it would be straightforward to include relevant stratifying variables as covariates in the propensity score estimation, our method provides for researchers to conduct retrospective analyses more consistently with the prospective experiment they would have designed if permitted. Namely, our method permits researchers to approximate a stratified randomized trial as opposed to the CRT otherwise obtainable by GPSM. We apply our proposed method to an analysis of how the number of children in a household affects systolic blood pressure in adults. We conduct a simulation study assessing how the relationship between response, treatment, and strata affect the performance of our method and compare the results to non-stratified GPSM.

High Resolution Treatment Effects Estimation: Uncovering Effect Heterogeneities with the Modified Causal Forest.

There is great demand for inferring causal effect heterogeneity and for open-source statistical software, which is readily available for practitioners. The mcf package is an open-source Python package that implements Modified Causal Forest (mcf), a causal machine learner. We replicate three well-known studies in the fields of epidemiology, medicine, and labor economics to demonstrate that our mcf package produces aggregate treatment effects, which align with previous results, and in addition, provides novel insights on causal effect heterogeneity. For all resolutions of treatment effects estimation, which can be identified, the mcf package provides inference. We conclude that the mcf constitutes a practical and extensive tool for a modern causal heterogeneous effects analysis.

Matching on poset-based average rank for multiple treatments to compare many unbalanced groups.

In this article, we propose an original matching procedure for multiple treatment frameworks based on partially ordered set theory (poset). In our proposal, called matching on poset-based average rank for multiple treatments (MARMoT), poset theory is used to summarize individuals' confounders and the relative average rank is used to balance confounders and match individuals in different treatment groups. This approach proves to be particularly useful for balancing confounders when the number of treatments considered is high. We apply our approach to the estimation of neighborhood effect on the fractures among older people in Turin (a city in northern Italy).

Optimal and ethical designs for hypothesis testing in multi-arm exponential trials.

Multi-arm clinical trials are complex experiments which involve several objectives. The demand for unequal allocations in a multi-treatment context is growing and adaptive designs are being increasingly used in several areas of medical research. For uncensored and censored exponential responses, we propose a constrained optimization approach in order to derive the design maximizing the power of the multivariate test of homogeneity, under a suitable ethical constraint. In the absence of censoring, we obtain a very simple closed-form solution that dominates the balanced design in terms of power and ethics. Our suggestion can also accommodate delayed responses and staggered entries, and can be implemented via response adaptive rules. While other targets proposed in the literature could present an unethical behavior, the suggested optimal allocation is frequently unbalanced by assigning more patients to the best treatment, both in the absence and presence of censoring. We evaluate the operating characteristics of our proposal theoretically and by simulations, also redesigning a real lung cancer trial, showing that the constrained optimal target guarantees very good performances in terms of ethical demands, power and estimation precision. Therefore, it is a valid and useful tool in designing clinical trials, especially oncological trials and clinical experiments for grave and novel infectious diseases, where the ethical concern is of primary importance.

A Markov chain approach for ranking treatments in network meta-analysis.

When interpreting the relative effects from a network meta-analysis (NMA), researchers are usually aware of the potential limitations that may render the results for some comparisons less useful or meaningless. In the presence of sufficient and appropriate data, some of these limitations (eg, risk of bias, small-study effects, publication bias) can be taken into account in the statistical analysis. Very often, though, the necessary data for applying these methods are missing and data limitations cannot be formally integrated into ranking. In addition, there are other important characteristics of the treatment comparisons that cannot be addressed within a statistical model but only through qualitative judgments; for example, the relevance of data to the research question, the plausibility of the assumptions, and so on. Here, we propose a new measure for treatment ranking called the Probability of Selecting a Treatment to Recommend (POST-R). We suggest that the order of treatments should represent the process of considering treatments for selection in clinical practice and we assign to each treatment a probability of being selected. This process can be considered as a Markov chain model that allows the end-users of NMA to select the most appropriate treatments based not only on the NMA results but also to information external to the NMA. In this way, we obtain rankings that can inform decision-making more efficiently as they represent not only the relative effects but also their potential limitations. We illustrate our approach using a NMA comparing treatments for chronic plaque psoriasis and we provide the Stata commands.

A simulation study to compare different estimation approaches for network meta-analysis and corresponding methods to evaluate the consistency assumption.

BACKGROUND: Network meta-analysis (NMA) is becoming increasingly popular in systematic reviews and health technology assessments. However, there is still ambiguity concerning the properties of the estimation approaches as well as for the methods to evaluate the consistency assumption. METHODS: We conducted a simulation study for networks with up to 5 interventions. We investigated the properties of different methods and give recommendations for practical application. We evaluated the performance of 3 different models for complex networks as well as corresponding global methods to evaluate the consistency assumption. The models are the frequentist graph-theoretical approach netmeta, the Bayesian mixed treatment comparisons (MTC) consistency model, and the MTC consistency model with stepwise removal of studies contributing to inconsistency identified in a leverage plot. RESULTS: We found that with a high degree of inconsistency none of the evaluated effect estimators produced reliable results, whereas with moderate or no inconsistency the estimator from the MTC consistency model and the netmeta estimator showed acceptable properties. We also saw a dependency on the amount of heterogeneity. Concerning the evaluated methods to evaluate the consistency assumption, none was shown to be suitable. CONCLUSIONS: Based on our results we recommend a pragmatic approach for practical application in NMA. The estimator from the netmeta approach or the estimator from the Bayesian MTC consistency model should be preferred. Since none of the methods to evaluate the consistency assumption showed satisfactory results, users should have a strong focus on the similarity as well as the homogeneity assumption.

The heterogeneous causal effects of neonatal care: a model of endogenous demand for multiple treatment options based on geographical access to care.

Neonatal units in the UK are organised into three levels, from highest Neonatal Intensive Care Unit (NICU), to Local Neonatal Unit (LNU) to lowest Special Care Unit (SCU). We model the endogenous treatment selection of neonatal care unit of birth to estimate the average and marginal treatment effects of different neonatal designations on infant mortality, length of stay and hospital costs. We use prognostic factors, survival and hospital care use data on all preterm births in England for 2014-2015, supplemented by national reimbursement tariffs and instrumental variables of travel time from a geographic information system. The data were consistent with a model of demand for preterm birth care driven by physical access. In-hospital mortality of infants born before 32 weeks was 8.5% overall, and 1.2 (95% CI: -0.7, 3.2) percentage points lower for live births in hospitals with NICU or SCU compared to those with an LNU according to instrumental variable estimates. We find imprecise differences in average total hospital costs by unit designation, with positive unobserved selection of those with higher unexplained absolute and incremental costs into NICU. Our results suggest a limited scope for improvement in infant mortality by increasing in-utero transfers based on unit designation alone.

Target-specific micronucleus induction by colon carcinogens: 2-Amino-1-methyl-6-phenylimidazo[4,5-b]pyridine and 1,2-dimethylhydrazine.

Genotoxicity occurring at the target organs of carcinogenesis is important for understanding the mechanisms of chemical carcinogenicity and also for setting of threshold estimation. In vivo gene mutations have been evaluated by transgenic animal models in which any organ can be targeted; however, the methodologies that have been applied to assess chromosomal aberrations including micronucleus induction, are organ restricted, (often to bone marrow hematopoietic cells, as a common example). For food and food-related chemicals, the digestive tract is the important target organ as it is the organ of first contact. In the present study, we used 2-amino-1-methyl-6-phenylimidazo[4,5-b]pyridine (PhIP) and 1,2-dimethylhydrazine (DMH) as model chemicals of carcinogens primarily targeting the colon. We evaluated the applicability of colon cells and hepatocytes, together with bone marrow cells, in the micronucleus assay. Both model chemicals induced micronuclei in the colon, which is the target organ of these carcinogens, after short- and long-term treatment(s). The results demonstrate the target specificity of micronucleus induction and the assay using organs other than bone marrow will play an important role in understanding the mechanism of carcinogenicity and predicting new carcinogenic agents.

Multicategory Outcome Weighted Margin-based Learning for Estimating Individualized Treatment Rules.

Due to heterogeneity for many chronic diseases, precise personalized medicine, also known as precision medicine, has drawn increasing attentions in the scientific community. One main goal of precision medicine is to develop the most effective tailored therapy for each individual patient. To that end, one needs to incorporate individual characteristics to detect a proper individual treatment rule (ITR), by which suitable decisions on treatment assignments can be made to optimize patients' clinical outcome. For binary treatment settings, outcome weighted learning (OWL) and several of its variations have been proposed recently to estimate the ITR by optimizing the conditional expected outcome given patients' information. However, for multiple treatment scenarios, it remains unclear how to use OWL effectively. It can be shown that some direct extensions of OWL for multiple treatments, such as one-versus-one and one-versus-rest methods, can yield suboptimal performance. In this paper, we propose a new learning method, named Multicategory Outcome weighted Margin-based Learning (MOML), for estimating ITR with multiple treatments. Our proposed method is very general and covers OWL as a special case. We show Fisher consistency for the estimated ITR, and establish convergence rate properties. Variable selection using the sparse l 1 penalty is also considered. Analysis of simulated examples and a type 2 diabetes mellitus observational study are used to demonstrate competitive performance of the proposed method.

Double robust estimation for multiple unordered treatments and clustered observations: Evaluating drug-eluting coronary artery stents.

Postmarket comparative effectiveness and safety analyses of therapeutic treatments typically involve large observational cohorts. We propose double robust machine learning estimation techniques for implantable medical device evaluations where there are more than two unordered treatments and patients are clustered in hospitals. This flexible approach also accommodates high-dimensional covariates drawn from clinical databases. The Massachusetts Data Analysis Center percutaneous coronary intervention cohort is used to assess the composite outcome of 10 drug-eluting stents among adults implanted with at least one drug-eluting stent in Massachusetts. We find remarkable discrimination between stents. A simulation study designed to mimic this coronary intervention cohort is also presented and produced similar results.

Matching algorithms for causal inference with multiple treatments.

Randomized clinical trials are ideal for estimating causal effects, because the distributions of background covariates are similar in expectation across treatment groups. When estimating causal effects using observational data, matching is a commonly used method to replicate the covariate balance achieved in a randomized clinical trial. Matching algorithms have a rich history dating back to the mid-1900s but have been used mostly to estimate causal effects between two treatment groups. When there are more than two treatments, estimating causal effects requires additional assumptions and techniques. We propose several novel matching algorithms that address the drawbacks of the current methods, and we use simulations to compare current and new methods. All of the methods display improved covariate balance in the matched sets relative to the prematched cohorts. In addition, we provide advice to investigators on which matching algorithms are preferred for different covariate distributions.

A comparison of arm-based and contrast-based models for network meta-analysis.

Differences between arm-based (AB) and contrast-based (CB) models for network meta-analysis (NMA) are controversial. We compare the CB model of Lu and Ades (2006), the AB model of Hong et al(2016), and two intermediate models, using hypothetical data and a selected real data set. Differences between models arise primarily from study intercepts being fixed effects in the Lu-Ades model but random effects in the Hong model, and we identify four key difference. (1) If study intercepts are fixed effects then only within-study information is used, but if they are random effects then between-study information is also used and can cause important bias. (2) Models with random study intercepts are suitable for deriving a wider range of estimands, eg, the marginal risk difference, when underlying risk is derived from the NMA data; but underlying risk is usually best derived from external data, and then models with fixed intercepts are equally good. (3) The Hong model allows treatment effects to be related to study intercepts, but the Lu-Ades model does not. (4) The Hong model is valid under a more relaxed missing data assumption, that arms (rather than contrasts) are missing at random, but this does not appear to reduce bias. We also describe an AB model with fixed study intercepts and a CB model with random study intercepts. We conclude that both AB and CB models are suitable for the analysis of NMA data, but using random study intercepts requires a strong rationale such as relating treatment effects to study intercepts.

Network meta-analysis of rare events using the Mantel-Haenszel method.

The Mantel-Haenszel (MH) method has been used for decades to synthesize data obtained from studies that compare two interventions with respect to a binary outcome. It has been shown to perform better than the inverse-variance method or Peto's odds ratio when data is sparse. Network meta-analysis (NMA) is increasingly used to compare the safety of medical interventions, synthesizing, eg, data on mortality or serious adverse events. In this setting, sparse data occur often and yet there is to-date, no extension of the MH method for the case of NMA. In this paper, we fill this gap by presenting a MH-NMA method for odds ratios. Similarly to the pairwise MH method, we assume common treatment effects. We implement our approach in R, and we provide freely available easy-to-use routines. We illustrate our approach using data from two previously published networks. We compare our results to those obtained from three other approaches to NMA, namely, NMA with noncentral hypergeometric likelihood, an inverse-variance NMA, and a Bayesian NMA with a binomial likelihood. We also perform simulations to assess the performance of our method and compare it with alternative methods. We conclude that our MH-NMA method offers a reliable approach to the NMA of binary outcomes, especially in the case or sparse data, and when the assumption of methodological and clinical homogeneity is justifiable.

Incorporating external evidence on between-trial heterogeneity in network meta-analysis.

In a network meta-analysis, between-study heterogeneity variances are often very imprecisely estimated because data are sparse, so standard errors of treatment differences can be highly unstable. External evidence can provide informative prior distributions for heterogeneity and, hence, improve inferences. We explore approaches for specifying informative priors for multiple heterogeneity variances in a network meta-analysis. First, we assume equal heterogeneity variances across all pairwise intervention comparisons (approach 1); incorporating an informative prior for the common variance is then straightforward. Models allowing unequal heterogeneity variances are more realistic; however, care must be taken to ensure implied variance-covariance matrices remain valid. We consider three strategies for specifying informative priors for multiple unequal heterogeneity variances. Initially, we choose different informative priors according to intervention comparison type and assume heterogeneity to be proportional across comparison types and equal within comparison type (approach 2). Next, we allow all heterogeneity variances in the network to differ, while specifying a common informative prior for each. We explore two different approaches to this: placing priors on variances and correlations separately (approach 3) or using an informative inverse Wishart distribution (approach 4). Our methods are exemplified through application to two network metaanalyses. Appropriate informative priors are obtained from previously published evidence-based distributions for heterogeneity. Relevant prior information on between-study heterogeneity can be incorporated into network meta-analyses, without needing to assume equal heterogeneity across treatment comparisons. The approaches proposed will be beneficial in sparse data sets and provide more appropriate intervals for treatment differences than those based on imprecise heterogeneity estimates.

Robust outcome weighted learning for optimal individualized treatment rules.

Personalized medicine has received increasing attentions among scientific communities in recent years. Because patients often have heterogenous responses to treatments, discovering individualized treatment rules (ITR) is an important component of precision medicine. To that end, one needs to develop a proper decision rule using patient-specific characteristics to maximize the expected clinical outcome, i.e. the optimal ITR. Recently, outcome weighted learning (OWL) has been proposed to estimate optimal ITR under a weighted classification framework. Since most of commonly used loss functions are unbounded, the resulting ITR may suffer similar effects of outliers as the corresponding classifiers. In this paper, we propose robust OWL (ROWL) to build more stable ITRs using a new family of bounded and non-convex loss functions. Moreover, we extend the proposed ROWL method to the multiple treatment setting under the angle-based classification structure. Our theoretical results show that ROWL is Fisher consistent, and can provide the estimation of rewards' ratios for the resulting ITRs. We develop an efficient difference of convex functions algorithm (DCA) to solve the corresponding nonconvex optimization problem. Through analysis of simulated examples and a real medical dataset, we demonstrate that the proposed ROWL method yields more competitive performance in terms of the empirical value function and the misclassification error than several existing methods.