Assessing the incremental value of new biomarkers based on OR rules.

In early detection of disease, a single biomarker often has inadequate classification performance, making it important to identify new biomarkers to combine with the existing marker for improved performance. A biologically natural method for combining biomarkers is to use logic rules, e.g., the OR/AND rules. In our motivating example of early detection of pancreatic cancer, the established biomarker CA19-9 is only present in a subclass of cancers; it is of interest to identify new biomarkers present in the other subclasses and declare disease when either marker is positive. While there has been research on developing biomarker combinations using the OR/AND rules, inference regarding the incremental value of the new marker within this framework is lacking and challenging due to statistical non-regularity. In this article, we aim to answer the inferential question of whether combining the new biomarker achieves better classification performance than using the existing biomarker alone, based on a nonparametrically estimated OR rule that maximizes the weighted average of sensitivity and specificity. We propose and compare various procedures for testing the incremental value of the new biomarker and constructing its confidence interval, using bootstrap, cross-validation, and a novel fuzzy p-value-based technique. We compare the performance of different methods via extensive simulation studies and apply them to the pancreatic cancer example.

Statistical Learning Methods to Determine Immune Correlates of Herpes Zoster in Vaccine Efficacy Trials.

Using Super Learner, a machine learning statistical method, we assessed varicella zoster virus-specific glycoprotein-based enzyme-linked immunosorbent assay (gpELISA) antibody titer as an individual-level signature of herpes zoster (HZ) risk in the Zostavax Efficacy and Safety Trial. Gender and pre- and postvaccination gpELISA titers had moderate ability to predict whether a 50-59 year old experienced HZ over 1-2 years of follow-up, with equal classification accuracy (cross-validated area under the receiver operator curve = 0.65) for vaccine and placebo recipients. Previous analyses suggested that fold-rise gpELISA titer is a statistical correlate of protection and supported the hypothesis that it is not a mechanistic correlate of protection. Our results also support this hypothesis.

Racial/Ethnic Differences in the Role of Childhood Adversities for Mental Disorders Among a Nationally Representative Sample of Adolescents.

BACKGROUND: Childhood adversities may play a key role in the onset of mental disorders and influence patterns by race/ethnicity. We examined the relations between childhood adversities and mental disorders by race/ethnicity in the National Comorbidity Survey-Adolescent Supplement. METHODS: Using targeted maximum likelihood estimation, a rigorous and flexible estimation procedure, we estimated the relationship of each adversity with mental disorders (behavior, distress, fear, and substance use), and estimated the distribution of disorders by race/ethnicity in the absence of adversities. Targeted maximum likelihood estimation addresses the challenge of a multidimensional exposure such as a set of adversities because it facilitates "learning" from the data the strength of the relationships between each adversity and outcome, incorporating any interactions or nonlinearity, specific to each racial/ethnic group. Cross-validation is used to select the best model without over fitting. RESULTS: Among adversities, physical abuse, emotional abuse, and sexual abuse had the strongest associations with mental disorders. Of all outcomes, behavior disorders were most strongly associated with adversities. Our comparisons of observed prevalences of mental disorders to estimates in the absence of adversities suggest lower prevalences of behavior disorders across all racial/ethnic groups. Estimates for distress disorders and substance use disorders varied in magnitude among groups, but some estimates were imprecise. Interestingly, results suggest that the adversities examined here do not play a major role in patterns of racial/ethnic differences in mental disorders. CONCLUSIONS: Although causal interpretation relies on assumptions, growing work on this topic suggests childhood adversities play an important role in mental disorder development in adolescents.

STATISTICAL INFERENCE FOR THE MEAN OUTCOME UNDER A POSSIBLY NON-UNIQUE OPTIMAL TREATMENT STRATEGY.

We consider challenges that arise in the estimation of the mean outcome under an optimal individualized treatment strategy defined as the treatment rule that maximizes the population mean outcome, where the candidate treatment rules are restricted to depend on baseline covariates. We prove a necessary and sufficient condition for the pathwise differentiability of the optimal value, a key condition needed to develop a regular and asymptotically linear (RAL) estimator of the optimal value. The stated condition is slightly more general than the previous condition implied in the literature. We then describe an approach to obtain root-n rate confidence intervals for the optimal value even when the parameter is not pathwise differentiable. We provide conditions under which our estimator is RAL and asymptotically efficient when the mean outcome is pathwise differentiable. We also outline an extension of our approach to a multiple time point problem. All of our results are supported by simulations.

The optimal dynamic treatment rule superlearner: considerations, performance, and application to criminal justice interventions.

The optimal dynamic treatment rule (ODTR) framework offers an approach for understanding which kinds of patients respond best to specific treatments - in other words, treatment effect heterogeneity. Recently, there has been a proliferation of methods for estimating the ODTR. One such method is an extension of the SuperLearner algorithm - an ensemble method to optimally combine candidate algorithms extensively used in prediction problems - to ODTRs. Following the ``causal roadmap," we causally and statistically define the ODTR and provide an introduction to estimating it using the ODTR SuperLearner. Additionally, we highlight practical choices when implementing the algorithm, including choice of candidate algorithms, metalearners to combine the candidates, and risk functions to select the best combination of algorithms. Using simulations, we illustrate how estimating the ODTR using this SuperLearner approach can uncover treatment effect heterogeneity more effectively than traditional approaches based on fitting a parametric regression of the outcome on the treatment, covariates and treatment-covariate interactions. We investigate the implications of choices in implementing an ODTR SuperLearner at various sample sizes. Our results show the advantages of: (1) including a combination of both flexible machine learning algorithms and simple parametric estimators in the library of candidate algorithms; (2) using an ensemble metalearner to combine candidates rather than selecting only the best-performing candidate; (3) using the mean outcome under the rule as a risk function. Finally, we apply the ODTR SuperLearner to the ``Interventions" study, an ongoing randomized controlled trial, to identify which justice-involved adults with mental illness benefit most from cognitive behavioral therapy to reduce criminal re-offending.

The Appalachia Mind Health Initiative (AMHI): a pragmatic randomized clinical trial of adjunctive internet-based cognitive behavior therapy for treating major depressive disorder among primary care patients.

BACKGROUND: Major depressive disorder (MDD) is a leading cause of disease morbidity. Combined treatment with antidepressant medication (ADM) plus psychotherapy yields a much higher MDD remission rate than ADM only. But 77% of US MDD patients are nonetheless treated with ADM only despite strong patient preferences for psychotherapy. This mismatch is due at least in part to a combination of cost considerations and limited availability of psychotherapists, although stigma and reluctance of PCPs to refer patients for psychotherapy are also involved. Internet-based cognitive behaviorial therapy (i-CBT) addresses all of these problems. METHODS: Enrolled patients (n = 3360) will be those who are beginning ADM-only treatment of MDD in primary care facilities throughout West Virginia, one of the poorest and most rural states in the country. Participating treatment providers and study staff at West Virginia University School of Medicine (WVU) will recruit patients and, after obtaining informed consent, administer a baseline self-report questionnaire (SRQ) and then randomize patients to 1 of 3 treatment arms with equal allocation: ADM only, ADM + self-guided i-CBT, and ADM + guided i-CBT. Follow-up SRQs will be administered 2, 4, 8, 13, 16, 26, 39, and 52 weeks after randomization. The trial has two primary objectives: to evaluate aggregate comparative treatment effects across the 3 arms and to estimate heterogeneity of treatment effects (HTE). The primary outcome will be episode remission based on a modified version of the patient-centered Remission from Depression Questionnaire (RDQ). The sample was powered to detect predictors of HTE that would increase the proportional remission rate by 20% by optimally assigning individuals as opposed to randomly assigning them into three treatment groups of equal size. Aggregate comparative treatment effects will be estimated using intent-to-treat analysis methods. Cumulative inverse probability weights will be used to deal with loss to follow-up. A wide range of self-report predictors of MDD heterogeneity of treatment effects based on previous studies will be included in the baseline SRQ. A state-of-the-art ensemble machine learning method will be used to estimate HTE. DISCUSSION: The study is innovative in using a rich baseline assessment and in having a sample large enough to carry out a well-powered analysis of heterogeneity of treatment effects. We anticipate finding that self-guided and guided i-CBT will both improve outcomes compared to ADM only. We also anticipate finding that the comparative advantages of adding i-CBT to ADM will vary significantly across patients. We hope to develop a stable individualized treatment rule that will allow patients and treatment providers to improve aggregate treatment outcomes by deciding collaboratively when ADM treatment should be augmented with i-CBT. TRIAL REGISTRATION: ClinicalTrials.gov NCT04120285 . Registered on October 19, 2019.

An omnibus non-parametric test of equality in distribution for unknown functions.

We present a novel family of nonparametric omnibus tests of the hypothesis that two unknown but estimable functions are equal in distribution when applied to the observed data structure. We developed these tests, which represent a generalization of the maximum mean discrepancy tests described in Gretton et al. [2006], using recent developments from the higher-order pathwise differentiability literature. Despite their complex derivation, the associated test statistics can be expressed rather simply as U-statistics. We study the asymptotic behavior of the proposed tests under the null hypothesis and under both fixed and local alternatives. We provide examples to which our tests can be applied and show that they perform well in a simulation study. As an important special case, our proposed tests can be used to determine whether an unknown function, such as the conditional average treatment effect, is equal to zero almost surely.

Toward computerized efficient estimation in infinite-dimensional models.

Despite the risk of misspecification they are tied to, parametric models continue to be used in statistical practice because they are simple and convenient to use. In particular, efficient estimation procedures in parametric models are easy to describe and implement. Unfortunately, the same cannot be said of semiparametric and nonparametric models. While the latter often reflect the level of available scientific knowledge more appropriately, performing efficient inference in these models is generally challenging. The efficient influence function is a key analytic object from which the construction of asymptotically efficient estimators can potentially be streamlined. However, the theoretical derivation of the efficient influence function requires specialized knowledge and is often a difficult task, even for experts. In this paper, we present a novel representation of the efficient influence function and describe a numerical procedure for approximating its evaluation. The approach generalizes the nonparametric procedures of Frangakis et al. (2015) and Luedtke et al. (2015) to arbitrary models. We present theoretical results to support our proposal, and illustrate the method in the context of several semiparametric problems. The proposed approach is an important step toward automating efficient estimation in general statistical models, thereby rendering more accessible the use of realistic models in statistical analyses.

Parametric-rate inference for one-sided differentiable parameters.

Suppose one has a collection of parameters indexed by a (possibly infinite dimensional) set. Given data generated from some distribution, the objective is to estimate the maximal parameter in this collection evaluated at the distribution that generated the data. This estimation problem is typically non-regular when the maximizing parameter is non-unique, and as a result standard asymptotic techniques generally fail in this case. We present a technique for developing parametric-rate confidence intervals for the quantity of interest in these non-regular settings. We show that our estimator is asymptotically efficient when the maximizing parameter is unique so that regular estimation is possible. We apply our technique to a recent example from the literature in which one wishes to report the maximal absolute correlation between a prespecified outcome and one of p predictors. The simplicity of our technique enables an analysis of the previously open case where p grows with sample size. Specifically, we only require that log p grows slower than n , where n is the sample size. We show that, unlike earlier approaches, our method scales to massive data sets: the point estimate and confidence intervals can be constructed in O(np) time.

Evaluating the impact of treating the optimal subgroup.

Suppose we have a binary treatment used to influence an outcome. Given data from an observational or controlled study, we wish to determine whether or not there exists some subset of observed covariates in which the treatment is more effective than the standard practice of no treatment. Furthermore, we wish to quantify the improvement in population mean outcome that will be seen if this subgroup receives treatment and the rest of the population remains untreated. We show that this problem is surprisingly challenging given how often it is an (at least implicit) study objective. Blindly applying standard techniques fails to yield any apparent asymptotic results, while using existing techniques to confront the non-regularity does not necessarily help at distributions where there is no treatment effect. Here, we describe an approach to estimate the impact of treating the subgroup which benefits from treatment that is valid in a nonparametric model and is able to deal with the case where there is no treatment effect. The approach is a slight modification of an approach that recently appeared in the individualized medicine literature.

Optimal Individualized Treatments in Resource-Limited Settings.

An individualized treatment rule (ITR) is a treatment rule which assigns treatments to individuals based on (a subset of) their measured covariates. An optimal ITR is the ITR which maximizes the population mean outcome. Previous works in this area have assumed that treatment is an unlimited resource so that the entire population can be treated if this strategy maximizes the population mean outcome. We consider optimal ITRs in settings where the treatment resource is limited so that there is a maximum proportion of the population which can be treated. We give a general closed-form expression for an optimal stochastic ITR in this resource-limited setting, and a closed-form expression for the optimal deterministic ITR under an additional assumption. We also present an estimator of the mean outcome under the optimal stochastic ITR in a large semiparametric model that at most places restrictions on the probability of treatment assignment given covariates. We give conditions under which our estimator is efficient among all regular and asymptotically linear estimators. All of our results are supported by simulations.

Super-Learning of an Optimal Dynamic Treatment Rule.

We consider the estimation of an optimal dynamic two time-point treatment rule defined as the rule that maximizes the mean outcome under the dynamic treatment, where the candidate rules are restricted to depend only on a user-supplied subset of the baseline and intermediate covariates. This estimation problem is addressed in a statistical model for the data distribution that is nonparametric, beyond possible knowledge about the treatment and censoring mechanisms. We propose data adaptive estimators of this optimal dynamic regime which are defined by sequential loss-based learning under both the blip function and weighted classification frameworks. Rather than a priori selecting an estimation framework and algorithm, we propose combining estimators from both frameworks using a super-learning based cross-validation selector that seeks to minimize an appropriate cross-validated risk. The resulting selector is guaranteed to asymptotically perform as well as the best convex combination of candidate algorithms in terms of loss-based dissimilarity under conditions. We offer simulation results to support our theoretical findings.

Targeted Learning of the Mean Outcome under an Optimal Dynamic Treatment Rule.

We consider estimation of and inference for the mean outcome under the optimal dynamic two time-point treatment rule defined as the rule that maximizes the mean outcome under the dynamic treatment, where the candidate rules are restricted to depend only on a user-supplied subset of the baseline and intermediate covariates. This estimation problem is addressed in a statistical model for the data distribution that is nonparametric beyond possible knowledge about the treatment and censoring mechanism. This contrasts from the current literature that relies on parametric assumptions. We establish that the mean of the counterfactual outcome under the optimal dynamic treatment is a pathwise differentiable parameter under conditions, and develop a targeted minimum loss-based estimator (TMLE) of this target parameter. We establish asymptotic linearity and statistical inference for this estimator under specified conditions. In a sequentially randomized trial the statistical inference relies upon a second-order difference between the estimator of the optimal dynamic treatment and the optimal dynamic treatment to be asymptotically negligible, which may be a problematic condition when the rule is based on multivariate time-dependent covariates. To avoid this condition, we also develop TMLEs and statistical inference for data adaptive target parameters that are defined in terms of the mean outcome under the estimate of the optimal dynamic treatment. In particular, we develop a novel cross-validated TMLE approach that provides asymptotic inference under minimal conditions, avoiding the need for any empirical process conditions. We offer simulation results to support our theoretical findings.

Discussion of Identification, Estimation and Approximation of Risk under Interventions that Depend on the Natural Value of Treatment Using Observational Data, by Jessica Young, Miguel Hernán, and James Robins.

Young, Hernán, and Robins consider the mean outcome under a dynamic intervention that may rely on the natural value of treatment. They first identify this value with a statistical target parameter, and then show that this statistical target parameter can also be identified with a causal parameter which gives the mean outcome under a stochastic intervention. The authors then describe estimation strategies for these quantities. Here we augment the authors' insightful discussion by sharing our experiences in situations where two causal questions lead to the same statistical estimand, or the newer problem that arises in the study of data adaptive parameters, where two statistical estimands can lead to the same estimation problem. Given a statistical estimation problem, we encourage others to always use a robust estimation framework where the data generating distribution truly belongs to the statistical model. We close with a discussion of a framework which has these properties.