SMART Binary: New Sample Size Planning Resources for SMART Studies with Binary Outcome Measurements.

Sequential Multiple-Assignment Randomized Trials (SMARTs) play an increasingly important role in psychological and behavioral health research. This experimental approach enables researchers to answer scientific questions about how to sequence and match interventions to the unique, changing needs of individuals. A variety of sample size planning resources for SMART studies have been developed, enabling researchers to plan SMARTs for addressing different types of scientific questions. However, relatively limited attention has been given to planning SMARTs with binary (dichotomous) outcomes, which often require higher sample sizes relative to continuous outcomes. Existing resources for estimating sample size requirements for SMARTs with binary outcomes do not consider the potential to improve power by including a baseline measurement and/or multiple repeated outcome measurements. The current paper addresses this issue by providing sample size planning simulation procedures and approximate formulas for two-wave repeated measures binary outcomes (i.e., two measurement times for the outcome variable, before and after intervention delivery). The simulation results agree well with the formulas. We also discuss how to use simulations to calculate power for studies with more than two outcome measurement occasions. Results show that having at least one repeated measurement of the outcome can substantially improve power under certain conditions.

Adaptive Designs in Implementation Science and Practice: Their Promise and the Need for Greater Understanding and Improved Communication.

The promise of adaptation and adaptive designs in implementation science has been hindered by the lack of clarity and precision in defining what it means to adapt, especially regarding the distinction between adaptive study designs and adaptive implementation strategies. To ensure a common language for science and practice, authors reviewed the implementation science literature and found that the term adaptive was used to describe interventions, implementation strategies, and trial designs. To provide clarity and offer recommendations for reporting and strengthening study design, we propose a taxonomy that describes fixed versus adaptive implementation strategies and implementation trial designs. To improve impact, (a) future implementation studies should prespecify implementation strategy core functions that in turn can be taught to and replicated by health system/community partners, (b) funders should support exploratory studies that refine and specify implementation strategies, and (c) investigators should systematically address design requirements and ethical considerations (e.g., randomization, blinding/masking) with health system/community partners. Expected final online publication date for the Annual Review of Public Health, Volume 45 is April 2024. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Estimating tree-based dynamic treatment regimes using observational data with restricted treatment sequences.

A dynamic treatment regime (DTR) is a sequence of decision rules that provide guidance on how to treat individuals based on their static and time-varying status. Existing observational data are often used to generate hypotheses about effective DTRs. A common challenge with observational data, however, is the need for analysts to consider "restrictions" on the treatment sequences. Such restrictions may be necessary for settings where (1) one or more treatment sequences that were offered to individuals when the data were collected are no longer considered viable in practice, (2) specific treatment sequences are no longer available, or (3) the scientific focus of the analysis concerns a specific type of treatment sequences (eg, "stepped-up" treatments). To address this challenge, we propose a restricted tree-based reinforcement learning (RT-RL) method that searches for an interpretable DTR with the maximum expected outcome, given a (set of) user-specified restriction(s), which specifies treatment options (at each stage) that ought not to be considered as part of the estimated tree-based DTR. In simulations, we evaluate the performance of RT-RL versus the standard approach of ignoring the partial data for individuals not following the (set of) restriction(s). The method is illustrated using an observational data set to estimate a two-stage stepped-up DTR for guiding the level of care placement for adolescents with substance use disorder.

Accounting for Context in Randomized Trials after Assignment.

Many preventive trials randomize individuals to intervention condition which is then delivered in a group setting. Other trials randomize higher levels, say organizations, and then use learning collaboratives comprised of multiple organizations to support improved implementation or sustainment. Other trials randomize or expand existing social networks and use key opinion leaders to deliver interventions through these networks. We use the term contextually driven to refer generally to such trials (traditionally referred to as clustering, where groups are formed either pre-randomization or post-randomization - i.e., a cluster-randomized trial), as these groupings or networks provide fixed or time-varying contexts that matter both theoretically and practically in the delivery of interventions. While such contextually driven trials can provide efficient and effective ways to deliver and evaluate prevention programs, they all require analytical procedures that take appropriate account of non-independence, something not always appreciated. Published analyses of many prevention trials have failed to take this into account. We discuss different types of contextually driven designs and then show that even small amounts of non-independence can inflate actual Type I error rates. This inflation leads to rejecting the null hypotheses too often, and erroneously leading us to conclude that there are significant differences between interventions when they do not exist. We describe a procedure to account for non-independence in the important case of a two-arm trial that randomizes units of individuals or organizations in both arms and then provides the active treatment in one arm through groups formed after assignment. We provide sample code in multiple programming languages to guide the analyst, distinguish diverse contextually driven designs, and summarize implications for multiple audiences.

Multitiered Systems of Support, Adaptive Interventions, and SMART Designs.

This article introduces the special section on adaptive interventions and sequential multiple-assignment randomized trial (SMART) research designs. In addition to describing the two accompanying articles, we discuss features of adaptive interventions (AIs) and describe the use of SMART design to optimize AIs in the context of multitiered systems of support (MTSS) and integrated MTSS. AI is a treatment delivery model that explicitly specifies how information about individuals should be used to decide which treatment to provide in practice. Principles that apply to the design of AIs may help to more clearly operationalize MTSS-based programs, improve their implementation in school settings, and increase their efficacy when used according to evidence-based decision rules. A SMART is a research design for developing and optimizing MTSS-based programs. We provide a running example of a SMART design to optimize an MTSS-aligned AI that integrates academic and behavioral interventions.

Change in Patient Outcomes After Augmenting a Low-level Implementation Strategy in Community Practices That Are Slow to Adopt a Collaborative Chronic Care Model: A Cluster Randomized Implementation Trial.

BACKGROUND: Implementation strategies are essential for promoting the uptake of evidence-based practices and for patients to receive optimal care. Yet strategies differ substantially in their intensity and feasibility. Lower-intensity strategies (eg, training and technical support) are commonly used but may be insufficient for all clinics. Limited research has examined the comparative effectiveness of augmentations to low-level implementation strategies for nonresponding clinics. OBJECTIVES: To compare 2 augmentation strategies for improving uptake of an evidence-based collaborative chronic care model (CCM) on 18-month outcomes for patients with depression at community-based clinics nonresponsive to lower-level implementation support. RESEARCH DESIGN: Providers initially received support using a low-level implementation strategy, Replicating Effective Programs (REP). After 6 months, nonresponsive clinics were randomized to add either external facilitation (REP+EF) or external and internal facilitation (REP+EF/IF). MEASURES: The primary outcome was patient 12-item short form survey (SF-12) mental health score at month 18. Secondary outcomes were patient health questionnaire (PHQ-9) depression score at month 18 and receipt of the CCM during months 6 through 18. RESULTS: Twenty-seven clinics were nonresponsive after 6 months of REP. Thirteen clinics (N=77 patients) were randomized to REP+EF and 14 (N=92) to REP+EF/IF. At 18 months, patients in the REP+EF/IF arm had worse SF-12 [diff, 8.38; 95% confidence interval (CI), 3.59-13.18] and PHQ-9 scores (diff, 1.82; 95% CI, -0.14 to 3.79), and lower odds of CCM receipt (odds ratio, 0.67; 95% CI, 0.30-1.49) than REP+EF patients. CONCLUSIONS: Patients at sites receiving the more intensive REP+EF/IF saw less improvement in mood symptoms at 18 months than those receiving REP+EF and were no more likely to receive the CCM. For community-based clinics, EF augmentation may be more feasible than EF/IF for implementing CCMs.

Lung Cancer Screening and Smoking Cessation Clinical Trials. SCALE (Smoking Cessation within the Context of Lung Cancer Screening) Collaboration.

National recommendations for lung cancer screening for former and current smokers aged 55-80 years with a 30-pack-year smoking history create demand to implement efficient and effective systems to offer smoking cessation on a large scale. These older, high-risk smokers differ from participants in past clinical trials of behavioral and pharmacologic interventions for tobacco dependence. There is a gap in knowledge about how best to design systems to extend reach and treatments to maximize smoking cessation in the context of lung cancer screening. Eight clinical trials, seven funded by the National Cancer Institute and one by the Veterans Health Administration, address this gap and form the SCALE (Smoking Cessation within the Context of Lung Cancer Screening) collaboration. This paper describes methodological issues related to the design of these clinical trials: clinical workflow, participant eligibility criteria, screening indication (baseline or annual repeat screen), assessment content, interest in stopping smoking, and treatment delivery method and dose, all of which will affect tobacco treatment outcomes. Tobacco interventions consider the "teachable moment" offered by lung cancer screening, how to incorporate positive and negative screening results, and coordination of smoking cessation treatment with clinical events associated with lung cancer screening. Unique data elements, such as perceived risk of lung cancer and costs of tobacco treatment, are of interest. Lung cancer screening presents a new and promising opportunity to reduce morbidity and mortality resulting from lung cancer that can be amplified by effective smoking cessation treatment. SCALE teamwork and collaboration promise to maximize knowledge gained from the clinical trials.

A Data Analysis Method for Using Longitudinal Binary Outcome Data from a SMART to Compare Adaptive Interventions.

Sequential multiple assignment randomized trials (SMARTs) are a useful and increasingly popular approach for gathering information to inform the construction of adaptive interventions to treat psychological and behavioral health conditions. Until recently, analysis methods for data from SMART designs considered only a single measurement of the outcome of interest when comparing the efficacy of adaptive interventions. Lu et al. proposed a method for considering repeated outcome measurements to incorporate information about the longitudinal trajectory of change. While their proposed method can be applied to many kinds of outcome variables, they focused mainly on linear models for normally distributed outcomes. Practical guidelines and extensions are required to implement this methodology with other types of repeated outcome measures common in behavioral research. In this article, we discuss implementation of this method with repeated binary outcomes. We explain how to compare adaptive interventions in terms of various summaries of repeated binary outcome measures, including average outcome (area under the curve) and delayed effects. The method is illustrated using an empirical example from a SMART study to develop an adaptive intervention for engaging alcohol- and cocaine-dependent patients in treatment. Monte Carlo simulations are provided to demonstrate the good performance of the proposed technique.

Sequential, Multiple Assignment, Randomized Trial Designs.

This JAMA Guide to Statistics and Methods explains sequential, multiple assignment, randomized trial (SMART) study designs, in which some or all participants are randomized at 2 or more decision points depending on the participant's response to prior treatment.

The Right Tool for the Job: Choosing Between Covariate-balancing and Generalized Boosted Model Propensity Scores.

Estimating the causal effect of an exposure (vs. some control) on an outcome using observational data often requires addressing the fact that exposed and control groups differ on pre-exposure characteristics that may be related to the outcome (confounders). Propensity score methods have long been used as a tool for adjusting for observed confounders in order to produce more valid causal effect estimates under the strong ignorability assumption. In this article, we compare two promising propensity score estimation methods (for time-invariant binary exposures) when assessing the average treatment effect on the treated: the generalized boosted models and covariate-balancing propensity scores, with the main objective to provide analysts with some rules-of-thumb when choosing between these two methods. We compare the methods across different dimensions including the presence of extraneous variables, the complexity of the relationship between exposure or outcome and covariates, and the residual variance in outcome and exposure. We found that when noncomplex relationships exist between outcome or exposure and covariates, the covariate-balancing method outperformed the boosted method, but under complex relationships, the boosted method performed better. We lay out criteria for when one method should be expected to outperform the other with no blanket statement on whether one method is always better than the other.

Implementing Effective Policy in a National Mental Health Reengagement Program for Veterans.

Policy is a powerful motivator of clinical change, but implementation success can depend on organizational characteristics. This article used validated measures of organizational resources, culture, and climate to predict uptake of a nationwide Veteran's Health Administration (VA) policy aimed at implementing Re-Engage, a brief care management program that reestablishes contact with veterans with serious mental illness lost to care. Patient care databases were used to identify 2738 veterans lost to care. Local recovery coordinators (LRCs) were to update disposition for 2738 veterans at 158 VA facilities and, as appropriate, facilitate a return to care. Multivariable regression was used to assess organizational culture and climate as predictors of early policy compliance (via LRC presence) and uptake at 6 months. Higher composite climate and culture scores were associated with higher odds of having a designated LRC but were not predictive of higher uptake. Sites with LRCs had significantly higher rates of updated documentation than sites without LRCs.

Comparing dynamic treatment regimes using repeated-measures outcomes: modeling considerations in SMART studies.

A dynamic treatment regime (DTR) is a sequence of decision rules, each of which recommends a treatment based on a patient's past and current health status. Sequential, multiple assignment, randomized trials (SMARTs) are multi-stage trial designs that yield data specifically for building effective DTRs. Modeling the marginal mean trajectories of a repeated-measures outcome arising from a SMART presents challenges, because traditional longitudinal models used for randomized clinical trials do not take into account the unique design features of SMART. We discuss modeling considerations for various forms of SMART designs, emphasizing the importance of considering the timing of repeated measures in relation to the treatment stages in a SMART. For illustration, we use data from three SMART case studies with increasing level of complexity, in autism, child attention deficit hyperactivity disorder, and adult alcoholism. In all three SMARTs, we illustrate how to accommodate the design features along with the timing of the repeated measures when comparing DTRs based on mean trajectories of the repeated-measures outcome.

Adaptive Interventions in Child and Adolescent Mental Health.

The treatment or prevention of child and adolescent mental health (CAMH) disorders often requires an individualized, sequential approach to intervention, whereby treatments (or prevention efforts) are adapted over time based on the youth's evolving status (e.g., early response, adherence). Adaptive interventions are intended to provide a replicable guide for the provision of individualized sequences of interventions in actual clinical practice. Recently, there has been great interest in the development of adaptive intervenions by investigators working in CAMH. The development of such replicable, real-world, individualized sequences of decision rules to guide the treatment or prevention of CAMH disorders represents an important "next step" in interventions research. The primary purpose of this special issue is to showcase some recent work on the science of adaptive interventions in CAMH. In this overview article, we review why individualized sequences of interventions are needed in CAMH, provide an introduction to adaptive interventions, briefly describe each of the articles included in this special issue, and describe some exciting areas of ongoing and future research. A hopeful outcome of this special issue is that it encourages other researchers in CAMH to pursue creative and significant research on adaptive interventions.

A Pilot SMART for Developing an Adaptive Treatment Strategy for Adolescent Depression.

This pilot study was conducted to assess the feasibility and acceptability of 4 adaptive treatment strategies (ATSs) for adolescent depression to plan for a subsequent full-scale clinical trial. The ATSs aim to address 2 questions that arise when personalizing treatment: (a) For adolescents treated with Interpersonal Psychotherapy for depressed adolescents (IPT-A; Mufson et al., 2004 ), at what time point should therapists make the determination that the adolescent is not likely to respond if the initial treatment plan is continued (week 4 or week 8)? (b) For adolescents who are judged to need their treatment augmented, should the therapist increase the number of IPT-A sessions or add pharmacotherapy (fluoxetine)? A 16-week pilot sequential multiple assignment randomized trial (SMART) was conducted with 32 adolescents (M age = 14.9) who had a diagnosis of major depressive disorder, dysthymic disorder, or depressive disorder not otherwise specified. Adolescents were primarily female (75%) and Caucasian (84.4%). Data regarding the feasibility and acceptability of the study and treatment procedures and treatment response rates were collected. Week 4 was the more feasible and acceptable decision point for assessing need for a change to treatment. Adolescents, parents, and therapists reported a range of attitudes about medication and more intensive therapy as treatment options. Results from the pilot study have yielded additional research questions for the full-scale SMART and will improve our ability to successfully conduct the trial.

Personalized Treatment of Mothers With ADHD and Their Young At-Risk Children: A SMART Pilot.

Young children of mothers with adult attention-deficit/hyperactivity disorder (ADHD) are at risk for ADHD by virtue of genetics and environmental factors. Moreover, parent ADHD is associated with maladaptive parenting and poor child behavioral treatment response. Thus, a combined approach consisting of behavioral parent training (BPT) and maternal stimulant medication (MSM) may be needed to effectively treat ADHD within families. However, providing combined BPT+MSM initially to all families may be unnecessarily burdensome because not all families likely need combined treatment. The purpose of this study is to examine how to combine, sequence, and personalize treatment for these multiplex families in order to yield benefits to both the parent and child, thereby impacting the course of child ADHD and disruptive behavior symptoms. This article presents our rationale for, design of, and preliminary experiences (based on 26 participants) with an ongoing pilot Sequential Multiple Assessment Randomized Trial (SMART) designed to answer questions regarding the feasibility and acceptability of study protocols and interventions. This article also describes how the subsequent full-scale SMART might change based on what is learned in the SMART pilot and illustrates how the full-scale SMART could be used to inform clinical decision making about how to combine, sequence, and personalize treatment for complex children and families in which a parent has ADHD.

Longitudinal Effects of Adaptive Interventions With a Speech-Generating Device in Minimally Verbal Children With ASD.

There are limited data on the effects of adaptive social communication interventions with a speech-generating device in autism. This study is the first to compare growth in communications outcomes among three adaptive interventions in school-age children with autism spectrum disorder (ASD) who are minimally verbal. Sixty-one children, ages 5-8 years, participated in a sequential, multiple-assignment randomized trial (SMART). All children received a developmental behavioral communication intervention: joint attention, symbolic play, engagement and regulation (JASP) with enhanced milieu teaching (EMT). The SMART included three 2-stage, 24-week adaptive interventions with different provisions of a speech-generating device (SGD) in the context of JASP+EMT. The first adaptive intervention, with no SGD, initially assigned JASP+EMT alone, then intensified JASP+EMT for slow responders. In the second adaptive intervention, slow responders to JASP+EMT were assigned JASP+EMT+SGD. The third adaptive intervention initially assigned JASP+EMT+SGD; then intensified JASP+EMT+SGD for slow responders. Analyses examined between-group differences in change in outcomes from baseline to Week 36. Verbal outcomes included spontaneous communicative utterances and novel words. Nonlinguistic communication outcomes included initiating joint attention and behavior regulation, and play. The adaptive intervention beginning with JASP+EMT+SGD was estimated as superior. There were significant (p < .05) between-group differences in change in spontaneous communicative utterances and initiating joint attention. School-age children with ASD who are minimally verbal make significant gains in communication outcomes with an adaptive intervention beginning with JASP+EMT+SGD. Future research should explore mediators and moderators of the adaptive intervention effects and second-stage intervention options that further capitalize on early gains in treatment.

Time-varying effect moderation using the structural nested mean model: estimation using inverse-weighted regression with residuals.

This article considers the problem of examining time-varying causal effect moderation using observational, longitudinal data in which treatment, candidate moderators, and possible confounders are time varying. The structural nested mean model (SNMM) is used to specify the moderated time-varying causal effects of interest in a conditional mean model for a continuous response given time-varying treatments and moderators. We present an easy-to-use estimator of the SNMM that combines an existing regression-with-residuals (RR) approach with an inverse-probability-of-treatment weighting (IPTW) strategy. The RR approach has been shown to identify the moderated time-varying causal effects if the time-varying moderators are also the sole time-varying confounders. The proposed IPTW+RR approach provides estimators of the moderated time-varying causal effects in the SNMM in the presence of an additional, auxiliary set of known and measured time-varying confounders. We use a small simulation experiment to compare IPTW+RR versus the traditional regression approach and to compare small and large sample properties of asymptotic versus bootstrap estimators of the standard errors for the IPTW+RR approach. This article clarifies the distinction between time-varying moderators and time-varying confounders. We illustrate the methodology in a case study to assess if time-varying substance use moderates treatment effects on future substance use.

Optimization of behavioral dynamic treatment regimens based on the sequential, multiple assignment, randomized trial (SMART).

BACKGROUND AND PURPOSE: A behavioral intervention is a program aimed at modifying behavior for the purpose of treating or preventing disease, promoting health, and/or enhancing well-being. Many behavioral interventions are dynamic treatment regimens, that is, sequential, individualized multicomponent interventions in which the intensity and/or type of treatment is varied in response to the needs and progress of the individual participant. The multiphase optimization strategy (MOST) is a comprehensive framework for development, optimization, and evaluation of behavioral interventions, including dynamic treatment regimens. The objective of optimization is to make dynamic treatment regimens more effective, efficient, scalable, and sustainable. An important tool for optimization of dynamic treatment regimens is the sequential, multiple assignment, randomized trial (SMART). The purpose of this article is to discuss how to develop optimized dynamic treatment regimens within the MOST framework. METHODS AND RESULTS: The article discusses the preparation, optimization, and evaluation phases of MOST. It is shown how MOST can be used to develop a dynamic treatment regimen to meet a prespecified optimization criterion. The SMART is an efficient experimental design for gathering the information needed to optimize a dynamic treatment regimen within MOST. One signature feature of the SMART is that randomization takes place at more than one point in time. CONCLUSION: MOST and SMART can be used to develop optimized dynamic treatment regimens that will have a greater public health impact.

What to do after smoking relapse? A sequential multiple assignment randomized trial of chronic care smoking treatments.

AIM: To compare effects of three post-relapse interventions on smoking abstinence. DESIGN: Sequential three-phase multiple assignment randomized trial (SMART). SETTING: Eighteen Wisconsin, USA, primary care clinics. PARTICIPANTS: A total of 1154 primary care patients (53.6% women, 81.2% White) interested in quitting smoking enrolled from 2015 to 2019; 582 relapsed and were randomized to relapse recovery treatment. INTERVENTIONS: In phase 1, patients received cessation counseling and 8 weeks nicotine patch. Those who relapsed and agreed were randomized to a phase 2 relapse recovery group: (1) reduction counseling + nicotine mini-lozenges + encouragement to quit starting 1 month post-randomization (preparation); (2) repeated encouragement to quit starting immediately post-randomization (recycling); or (3) advice to call the tobacco quitline (control). The first two groups could opt into phase 3 new quit treatment [8 weeks nicotine patch + mini-lozenges plus randomization to two treatment factors (skill training and supportive counseling) in a 2 × 2 design]. Phase 2 and 3 interventions lasted ≤ 15 months. MEASUREMENTS: The study was powered to compare each active phase 2 treatment with the control on the primary outcome: biochemically confirmed 7-day point-prevalence abstinence 14 months post initiating phase 2 relapse recovery treatment. Exploratory analyses tested for phase 3 counseling factor effects. FINDINGS: Neither skill training nor supportive counseling (each on versus off) increased 14-month abstinence rates; skills on versus off 9.3% (14/151) versus 5.2% (8/153), P = 0.19; support on versus off 6.6% (10/152) versus 7.9% (12/152), P = 0.73. Phase 2 preparation did not produce higher 14-month abstinence rates than quitline referral; 3.6% (8/220) versus 2.1% [3/145; risk difference = 1.5%, 95% confidence interval (CI) = -1.8-5.0%, odds ratio (OR) = 1.8, 95% CI = 0.5-6.9]. Recycling, however, produced higher abstinence rates than quitline referral; 6.9% (15/217) versus 2.1% (three of 145; risk difference, 4.8%, 95% CI = 0.7-8.9%, OR = 3.5, 95% CI = 1.0-12.4). Recycling produced greater entry into new quit treatment than preparation: 83.4% (181/217) versus 55.9% (123/220), P < 0.0001. CONCLUSIONS: Among people interested in quitting smoking, immediate encouragement post-relapse to enter a new round of smoking cessation treatment ('recycling') produced higher probability of abstinence than tobacco quitline referral. Recycling produced higher rates of cessation treatment re-engagement than did preparation/cutting down using more intensive counseling and pharmacotherapy.

A tutorial on propensity score estimation for multiple treatments using generalized boosted models.

The use of propensity scores to control for pretreatment imbalances on observed variables in non-randomized or observational studies examining the causal effects of treatments or interventions has become widespread over the past decade. For settings with two conditions of interest such as a treatment and a control, inverse probability of treatment weighted estimation with propensity scores estimated via boosted models has been shown in simulation studies to yield causal effect estimates with desirable properties. There are tools (e.g., the twang package in R) and guidance for implementing this method with two treatments. However, there is not such guidance for analyses of three or more treatments. The goals of this paper are twofold: (1) to provide step-by-step guidance for researchers who want to implement propensity score weighting for multiple treatments and (2) to propose the use of generalized boosted models (GBM) for estimation of the necessary propensity score weights. We define the causal quantities that may be of interest to studies of multiple treatments and derive weighted estimators of those quantities. We present a detailed plan for using GBM to estimate propensity scores and using those scores to estimate weights and causal effects. We also provide tools for assessing balance and overlap of pretreatment variables among treatment groups in the context of multiple treatments. A case study examining the effects of three treatment programs for adolescent substance abuse demonstrates the methods.