Intervention Optimization: A Paradigm Shift and Its Potential Implications for Clinical Psychology.

To build a coherent knowledge base about what psychological intervention strategies work, develop interventions that have positive societal impact, and maintain and increase this impact over time, it is necessary to replace the classical treatment package research paradigm. The multiphase optimization strategy (MOST) is an alternative paradigm that integrates ideas from behavioral science, engineering, implementation science, economics, and decision science. MOST enables optimization of interventions to strategically balance effectiveness, affordability, scalability, and efficiency. In this review we provide an overview of MOST, discuss several experimental designs that can be used in intervention optimization, consider how the investigator can use experimental results to select components for inclusion in the optimized intervention, discuss the application of MOST in implementation science, and list future issues in this rapidly evolving field. We highlight the feasibility of adopting this new research paradigm as well as its potential to hasten the progress of psychological intervention science.

Sample size considerations for micro-randomized trials with binary proximal outcomes.

Micro-randomized trials (MRTs) are a novel experimental design for developing mobile health interventions. Participants are repeatedly randomized in an MRT, resulting in longitudinal data with time-varying treatments. Causal excursion effects are the main quantities of interest in MRT primary and secondary analyses. We consider MRTs where the proximal outcome is binary and the randomization probability is constant or time-varying but not data-dependent. We develop a sample size formula for detecting a nonzero marginal excursion effect. We prove that the formula guarantees power under a set of working assumptions. We demonstrate via simulation that violations of certain working assumptions do not affect the power, and for those that do, we point out the direction in which the power changes. We then propose practical guidelines for using the sample size formula. As an illustration, the formula is used to size an MRT on interventions for excessive drinking. The sample size calculator is implemented in R package MRTSampleSizeBinary and an interactive R Shiny app. This work can be used in trial planning for a wide range of MRTs with binary proximal outcomes.

BATCH POLICY LEARNING IN AVERAGE REWARD MARKOV DECISION PROCESSES.

We consider the batch (off-line) policy learning problem in the infinite horizon Markov Decision Process. Motivated by mobile health applications, we focus on learning a policy that maximizes the long-term average reward. We propose a doubly robust estimator for the average reward and show that it achieves semiparametric efficiency. Further we develop an optimization algorithm to compute the optimal policy in a parameterized stochastic policy class. The performance of the estimated policy is measured by the difference between the optimal average reward in the policy class and the average reward of the estimated policy and we establish a finite-sample regret guarantee. The performance of the method is illustrated by simulation studies and an analysis of a mobile health study promoting physical activity.

Linear mixed models with endogenous covariates: modeling sequential treatment effects with application to a mobile health study.

Mobile health is a rapidly developing field in which behavioral treatments are delivered to individuals via wearables or smartphones to facilitate health-related behavior change. Micro-randomized trials (MRT) are an experimental design for developing mobile health interventions. In an MRT the treatments are randomized numerous times for each individual over course of the trial. Along with assessing treatment effects, behavioral scientists aim to understand between-person heterogeneity in the treatment effect. A natural approach is the familiar linear mixed model. However, directly applying linear mixed models is problematic because potential moderators of the treatment effect are frequently endogenous-that is, may depend on prior treatment. We discuss model interpretation and biases that arise in the absence of additional assumptions when endogenous covariates are included in a linear mixed model. In particular, when there are endogenous covariates, the coefficients no longer have the customary marginal interpretation. However, these coefficients still have a conditional-on-the-random-effect interpretation. We provide an additional assumption that, if true, allows scientists to use standard software to fit linear mixed model with endogenous covariates, and person-specific predictions of effects can be provided. As an illustration, we assess the effect of activity suggestion in the HeartSteps MRT and analyze the between-person treatment effect heterogeneity.

Efficacy of Contextually Tailored Suggestions for Physical Activity: A Micro-randomized Optimization Trial of HeartSteps.

BACKGROUND: HeartSteps is an mHealth intervention that encourages regular walking via activity suggestions tailored to the individuals' current context. PURPOSE: We conducted a micro-randomized trial (MRT) to evaluate the efficacy of HeartSteps' activity suggestions to optimize the intervention. METHODS: We conducted a 6-week MRT with 44 adults. Contextually tailored suggestions could be delivered up to five times per day at user-selected times. At each of these five times, for each participant on each day of the study, HeartSteps randomized whether to provide an activity suggestion, and, if so, whether to provide a walking or an antisedentary suggestion. We used a centered and weighted least squares method to analyze the effect of suggestions on the 30-min step count following suggestion randomization. RESULTS: Averaging over study days and types of activity suggestions, delivering a suggestion versus no suggestion increased the 30-min step count by 14% (p = .06), 35 additional steps over the 253-step average. The effect was not evenly distributed in time. Providing any type of suggestion versus no suggestion initially increased the step count by 66% (167 steps; p < .01), but this effect diminished over time. Averaging over study days, delivering a walking suggestion versus no suggestion increased the average step count by 24% (59 steps; p = .02). This increase was greater at the start of study (107% or 271 additional steps; p < .01), but decreased over time. Antisedentary suggestions had no detectable effect on the 30-min step count. CONCLUSION: Contextually tailored walking suggestions are a promising way of initiating bouts of walking throughout the day. CLINICAL TRIAL INFORMATION: This study was registered on ClinicalTrials.gov number NCT03225521.

From hype to reality: data science enabling personalized medicine.

BACKGROUND: Personalized, precision, P4, or stratified medicine is understood as a medical approach in which patients are stratified based on their disease subtype, risk, prognosis, or treatment response using specialized diagnostic tests. The key idea is to base medical decisions on individual patient characteristics, including molecular and behavioral biomarkers, rather than on population averages. Personalized medicine is deeply connected to and dependent on data science, specifically machine learning (often named Artificial Intelligence in the mainstream media). While during recent years there has been a lot of enthusiasm about the potential of 'big data' and machine learning-based solutions, there exist only few examples that impact current clinical practice. The lack of impact on clinical practice can largely be attributed to insufficient performance of predictive models, difficulties to interpret complex model predictions, and lack of validation via prospective clinical trials that demonstrate a clear benefit compared to the standard of care. In this paper, we review the potential of state-of-the-art data science approaches for personalized medicine, discuss open challenges, and highlight directions that may help to overcome them in the future. CONCLUSIONS: There is a need for an interdisciplinary effort, including data scientists, physicians, patient advocates, regulatory agencies, and health insurance organizations. Partially unrealistic expectations and concerns about data science-based solutions need to be better managed. In parallel, computational methods must advance more to provide direct benefit to clinical practice.

Just-in-Time Adaptive Interventions (JITAIs) in Mobile Health: Key Components and Design Principles for Ongoing Health Behavior Support.

Background: The just-in-time adaptive intervention (JITAI) is an intervention design aiming to provide the right type/amount of support, at the right time, by adapting to an individual's changing internal and contextual state. The availability of increasingly powerful mobile and sensing technologies underpins the use of JITAIs to support health behavior, as in such a setting an individual's state can change rapidly, unexpectedly, and in his/her natural environment. Purpose: Despite the increasing use and appeal of JITAIs, a major gap exists between the growing technological capabilities for delivering JITAIs and research on the development and evaluation of these interventions. Many JITAIs have been developed with minimal use of empirical evidence, theory, or accepted treatment guidelines. Here, we take an essential first step towards bridging this gap. Methods: Building on health behavior theories and the extant literature on JITAIs, we clarify the scientific motivation for JITAIs, define their fundamental components, and highlight design principles related to these components. Examples of JITAIs from various domains of health behavior research are used for illustration. Conclusions: As we enter a new era of technological capacity for delivering JITAIs, it is critical that researchers develop sophisticated and nuanced health behavior theories capable of guiding the construction of such interventions. Particular attention has to be given to better understanding the implications of providing timely and ecologically sound support for intervention adherence and retention.

Sample size calculations for micro-randomized trials in mHealth.

The use and development of mobile interventions are experiencing rapid growth. In "just-in-time" mobile interventions, treatments are provided via a mobile device, and they are intended to help an individual make healthy decisions 'in the moment,' and thus have a proximal, near future impact. Currently, the development of mobile interventions is proceeding at a much faster pace than that of associated data science methods. A first step toward developing data-based methods is to provide an experimental design for testing the proximal effects of these just-in-time treatments. In this paper, we propose a 'micro-randomized' trial design for this purpose. In a micro-randomized trial, treatments are sequentially randomized throughout the conduct of the study, with the result that each participant may be randomized at the 100s or 1000s of occasions at which a treatment might be provided. Further, we develop a test statistic for assessing the proximal effect of a treatment as well as an associated sample size calculator. We conduct simulation evaluations of the sample size calculator in various settings. Rules of thumb that might be used in designing a micro-randomized trial are discussed. This work is motivated by our collaboration on the HeartSteps mobile application designed to increase physical activity. Copyright © 2015 John Wiley & Sons, Ltd.

Treatment Sequencing for Childhood ADHD: A Multiple-Randomization Study of Adaptive Medication and Behavioral Interventions.

Behavioral and pharmacological treatments for children with attention deficit/hyperactivity disorder (ADHD) were evaluated to address whether endpoint outcomes are better depending on which treatment is initiated first and, in case of insufficient response to initial treatment, whether increasing dose of initial treatment or adding the other treatment modality is superior. Children with ADHD (ages 5-12, N = 146, 76% male) were treated for 1 school year. Children were randomized to initiate treatment with low doses of either (a) behavioral parent training (8 group sessions) and brief teacher consultation to establish a Daily Report Card or (b) extended-release methylphenidate (equivalent to .15 mg/kg/dose bid). After 8 weeks or at later monthly intervals as necessary, insufficient responders were rerandomized to secondary interventions that either increased the dose/intensity of the initial treatment or added the other treatment modality, with adaptive adjustments monthly as needed to these secondary treatments. The group beginning with behavioral treatment displayed significantly lower rates of observed classroom rule violations (the primary outcome) at study endpoint and tended to have fewer out-of-class disciplinary events. Further, adding medication secondary to initial behavior modification resulted in better outcomes on the primary outcomes and parent/teacher ratings of oppositional behavior than adding behavior modification to initial medication. Normalization rates on teacher and parent ratings were generally high. Parents who began treatment with behavioral parent training had substantially better attendance than those assigned to receive training following medication. Beginning treatment with behavioral intervention produced better outcomes overall than beginning treatment with medication.

Benchmarking participation of Canadian university health sciences librarians in systematic reviews.

This study describes the current state of Canadian university health sciences librarians' knowledge about, training needs for, and barriers to participating in systematic reviews (SRs). A convenience sample of Canadian librarians was surveyed. Over half of the librarians who had participated in SRs acknowledged participating in a traditional librarian role (e.g., search strategy developer); less than half indicated participating in any one nontraditional librarian role (e.g., data extractor). Lack of time and insufficient training were the most frequently reported barriers to participating in SRs. The findings provide a benchmark for tracking changes in Canadian university health sciences librarians' participation in SRs.

Time-varying effect moderation using the structural nested mean model: estimation using inverse-weighted regression with residuals.

This article considers the problem of examining time-varying causal effect moderation using observational, longitudinal data in which treatment, candidate moderators, and possible confounders are time varying. The structural nested mean model (SNMM) is used to specify the moderated time-varying causal effects of interest in a conditional mean model for a continuous response given time-varying treatments and moderators. We present an easy-to-use estimator of the SNMM that combines an existing regression-with-residuals (RR) approach with an inverse-probability-of-treatment weighting (IPTW) strategy. The RR approach has been shown to identify the moderated time-varying causal effects if the time-varying moderators are also the sole time-varying confounders. The proposed IPTW+RR approach provides estimators of the moderated time-varying causal effects in the SNMM in the presence of an additional, auxiliary set of known and measured time-varying confounders. We use a small simulation experiment to compare IPTW+RR versus the traditional regression approach and to compare small and large sample properties of asymptotic versus bootstrap estimators of the standard errors for the IPTW+RR approach. This article clarifies the distinction between time-varying moderators and time-varying confounders. We illustrate the methodology in a case study to assess if time-varying substance use moderates treatment effects on future substance use.

A multiple imputation strategy for sequential multiple assignment randomized trials.

Sequential multiple assignment randomized trials (SMARTs) are increasingly being used to inform clinical and intervention science. In a SMART, each patient is repeatedly randomized over time. Each randomization occurs at a critical decision point in the treatment course. These critical decision points often correspond to milestones in the disease process or other changes in a patient's health status. Thus, the timing and number of randomizations may vary across patients and depend on evolving patient-specific information. This presents unique challenges when analyzing data from a SMART in the presence of missing data. This paper presents the first comprehensive discussion of missing data issues typical of SMART studies: we describe five specific challenges and propose a flexible imputation strategy to facilitate valid statistical estimation and inference using incomplete data from a SMART. To illustrate these contributions, we consider data from the Clinical Antipsychotic Trial of Intervention and Effectiveness, one of the most well-known SMARTs to date.

Optimizing an adaptive digital oral health intervention for promoting oral self-care behaviors: Micro-randomized trial protocol.

Dental disease continues to be one of the most prevalent chronic diseases in the United States. Although oral self-care behaviors (OSCB), involving systematic twice-a-day tooth brushing, can prevent dental disease, this basic behavior is not sufficiently practiced. Recent advances in digital technology offer tremendous potential for promoting OSCB by delivering Just-In-Time Adaptive Interventions (JITAIs)- interventions that leverage dynamic information about the person's state and context to effectively prompt them to engage in a desired behavior in real-time, real-world settings. However, limited research attention has been given to systematically investigating how to best prompt individuals to engage in OSCB in daily life, and under what conditions prompting would be most beneficial. This paper describes the protocol for a Micro-Randomized Trial (MRT) to inform the development of a JITAI for promoting ideal OSCB, namely, brushing twice daily, for two minutes each time, in all four dental quadrants (i.e., 2x2x4). Sensors within an electric toothbrush (eBrush) will be used to track OSCB and a matching mobile app (Oralytics) will deliver on-demand feedback and educational information. The MRT will micro-randomize participants twice daily (morning and evening) to either (a) a prompt (push notification) containing one of several theoretically grounded engagement strategies or (b) no prompt. The goal is to investigate whether, what type of, and under what conditions prompting increases engagement in ideal OSCB. The results will build the empirical foundation necessary to develop an optimized JITAI that will be evaluated relative to a suitable control in a future randomized controlled trial.

Subgroups analysis when treatment and moderators are time-varying.

Prevention scientists are often interested in understanding characteristics of participants that are predictive of treatment effects because these characteristics can be used to inform the types of individuals who benefit more or less from treatment or prevention programs. Often, effect moderation questions are examined using subgroups analysis or, equivalently, using covariate × treatment interactions in the context of regression analysis. This article focuses on conceptualizing and examining causal effect moderation in longitudinal settings in which both treatment and the putative moderators are time-varying. Studying effect moderation in the time-varying setting helps identify which individuals will benefit more or less from additional treatment services on the basis of both individual characteristics and their evolving outcomes, symptoms, severity, and need. Examining effect moderation in these longitudinal settings, however, is difficult because moderators of future treatment may themselves be affected by prior treatment (for example, future moderators may be mediators of prior treatment). This article introduces moderated intermediate causal effects in the time-varying setting, describes how they are part of Robins' Structural Nested Mean Model, discusses two problems with using a traditional regression approach to estimate these effects, and describes a new approach (a two-stage regression estimator) to estimate these effects. The methodology is illustrated using longitudinal data to examine the time-varying effects of receiving community-based substance abuse treatment as a function of time-varying severity (or need).

Assessing the Impact of Context Inference Error and Partial Observability on RL Methods for Just-In-Time Adaptive Interventions.

Just-in-Time Adaptive Interventions (JITAIs) are a class of personalized health interventions developed within the behavioral science community. JITAIs aim to provide the right type and amount of support by iteratively selecting a sequence of intervention options from a pre-defined set of components in response to each individual's time varying state. In this work, we explore the application of reinforcement learning methods to the problem of learning intervention option selection policies. We study the effect of context inference error and partial observability on the ability to learn effective policies. Our results show that the propagation of uncertainty from context inferences is critical to improving intervention efficacy as context uncertainty increases, while policy gradient algorithms can provide remarkable robustness to partially observed behavioral state information.