The Digital Therapeutics Real-World Evidence Framework: An Approach for Guiding Evidence-Based Digital Therapeutics Design, Development, Testing, and Monitoring.

Digital therapeutics (DTx) are a promising way to provide safe, effective, accessible, sustainable, scalable, and equitable approaches to advance individual and population health. However, developing and deploying DTx is inherently complex in that DTx includes multiple interacting components, such as tools to support activities like medication adherence, health behavior goal-setting or self-monitoring, and algorithms that adapt the provision of these according to individual needs that may change over time. While myriad frameworks exist for different phases of DTx development, no single framework exists to guide evidence production for DTx across its full life cycle, from initial DTx development to long-term use. To fill this gap, we propose the DTx real-world evidence (RWE) framework as a pragmatic, iterative, milestone-driven approach for developing DTx. The DTx RWE framework is derived from the 4-phase development model used for behavioral interventions, but it includes key adaptations that are specific to the unique characteristics of DTx. To ensure the highest level of fidelity to the needs of users, the framework also incorporates real-world data (RWD) across the entire life cycle of DTx development and use. The DTx RWE framework is intended for any group interested in developing and deploying DTx in real-world contexts, including those in industry, health care, public health, and academia. Moreover, entities that fund research that supports the development of DTx and agencies that regulate DTx might find the DTx RWE framework useful as they endeavor to improve how DTxcan advance individual and population health.

Effect-Invariant Mechanisms for Policy Generalization.

Policy learning is an important component of many real-world learning systems. A major challenge in policy learning is how to adapt efficiently to unseen environments or tasks. Recently, it has been suggested to exploit invariant conditional distributions to learn models that generalize better to unseen environments. However, assuming invariance of entire conditional distributions (which we call full invariance) may be too strong of an assumption in practice. In this paper, we introduce a relaxation of full invariance called effect-invariance (e-invariance for short) and prove that it is sufficient, under suitable assumptions, for zero-shot policy generalization. We also discuss an extension that exploits e-invariance when we have a small sample from the test environment, enabling few-shot policy generalization. Our work does not assume an underlying causal graph or that the data are generated by a structural causal model; instead, we develop testing procedures to test e-invariance directly from data. We present empirical results using simulated data and a mobile health intervention dataset to demonstrate the effectiveness of our approach.

Getting cozy with causality: Advances to the causal pathway diagramming method to enhance implementation precision.

Background: Implementation strategies are theorized to work well when carefully matched to implementation determinants and when factors-preconditions, moderators, etc.-that influence strategy effectiveness are prospectively identified and addressed. Existing methods for strategy selection are either imprecise or require significant technical expertise and resources, undermining their utility. This article outlines refinements to causal pathway diagrams (CPDs), a method for articulating the causal process through which implementation strategies work and offers illustrations of their use. Method: CPDs are a visualization tool to represent an implementation strategy, its mechanism(s) (i.e., the processes through which a strategy is thought to operate), determinants it is intended to address, factors that may impede or facilitate its effectiveness, and the series of outcomes that should be expected if the strategy is operating as intended. We offer principles for constructing CPDs and describe their key functions. Results: Applications of the CPD method by study teams from two National Institute of Health-funded Implementation Science Centers and a research grant are presented. These include the use of CPDs to (a) match implementation strategies to determinants, (b) understand the conditions under which an implementation strategy works, and (c) develop causal theories of implementation strategies. Conclusions: CPDs offer a novel method for implementers to select, understand, and improve the effectiveness of implementation strategies. They make explicit theoretical assumptions about strategy operation while supporting practical planning. Early applications have led to method refinements and guidance for the field.

Advances to the Causal Pathway Diagramming Method to Enhance Implementation Precision Plain Language Summary Implementation strategies often fail to produce meaningful improvements in the outcomes we hope to impact. Better tools for choosing, designing, and evaluating implementation strategies may improve their performance. We developed a tool, causal pathway diagrams (CPD), to visualize and describe how implementation strategies are expected to work. In this article, we describe refinements to the CPD tool and accompanying approach. We use real illustrations to show how CPDs can be used to improve how to match strategies to barriers, understand the conditions in which those strategies work best, and develop generalizable theories describing how implementation strategies work. CPDs can serve as both a practical and scientific tool to improve the planning, deployment, and evaluation of implementation strategies. We demonstrate the range of ways that CPDs are being used, from a highly practical tool to improve implementation practice to a scientific approach to advance testing and theorizing about implementation strategies.

Text Messages to Promote Physical Activity in Patients With Cardiovascular Disease: A Micro-Randomized Trial of a Just-In-Time Adaptive Intervention.

BACKGROUND: Text messages may enhance physical activity levels in patients with cardiovascular disease, including those enrolled in cardiac rehabilitation. However, the independent and long-term effects of text messages remain uncertain. METHODS: The VALENTINE study (Virtual Application-supported Environment to Increase Exercise) was a micro-randomized trial that delivered text messages through a smartwatch (Apple Watch or Fitbit Versa) to participants initiating cardiac rehabilitation. Participants were randomized 4× per day over 6-months to receive no text message or a message encouraging low-level physical activity. Text messages were tailored on contextual factors (eg, weather). Our primary outcome was step count 60 minutes following a text message, and we used a centered and weighted least squares mean method to estimate causal effects. Given potential measurement differences between devices determined a priori, data were assessed separately for Apple Watch and Fitbit Versa users over 3 time periods corresponding to the initiation (0-30 days), maintenance (31-120 days), and completion (121-182 days) of cardiac rehabilitation. RESULTS: One hundred eight participants were included with 70 552 randomizations over 6 months; mean age was 59.5 (SD, 10.7) years with 36 (32.4%) female and 68 (63.0%) Apple Watch participants. For Apple Watch participants, text messages led to a trend in increased step count by 10% in the 60-minutes following a message during days 1 to 30 (95% CI, -1% to +20%), with no effect from days 31 to 120 (+1% [95% CI, -4% to +5%]), and a significant 6% increase during days 121 to 182 (95% CI, +0% to +11%). For Fitbit users, text messages significantly increased step count by 17% (95% CI, +7% to +28%) in the 60-minutes following a message in the first 30 days of the study with no effect subsequently. CONCLUSIONS: In patients undergoing cardiac rehabilitation, contextually tailored text messages may increase physical activity, but this effect varies over time and by device. REGISTRATION: URL: https://www.clinicaltrials.gov; Unique identifier: NCT04587882.

Online model selection by learning how compositional kernels evolve.

Motivated by the need for efficient, personalized learning in mobile health, we investigate the problem of online compositional kernel selection for multi-task Gaussian Process regression. Existing composition selection methods do not satisfy our strict criteria in health; selection must occur quickly, and the selected kernels must maintain the appropriate level of complexity, sparsity, and stability as data arrives online. We introduce the Kernel Evolution Model (KEM), a generative process on how to evolve kernel compositions in a way that manages the bias-variance trade-off as we observe more data about a user. Using pilot data, we learn a set of kernel evolutions that can be used to quickly select kernels for new test users. KEM reliably selects high-performing kernels for a range of synthetic and real data sets, including two health data sets.