A reinforcement learning based algorithm for personalization of digital, just-in-time, adaptive interventions.

Suboptimal health related behaviors and habits; and resulting chronic diseases are responsible for majority of deaths globally. Studies show that providing personalized support to patients yield improved results by preventing and/or timely treatment of these problems. Digital, just-in-time and adaptive interventions are mobile phone-based notifications that are being utilized to support people wherever and whenever necessary in coping with their health problems. In this research, we propose a reinforcement learning-based mechanism to personalize interventions in terms of timing, frequency and preferred type(s). We simultaneously employ two reinforcement learning models, namely intervention-selection and opportune-moment-identification; capturing and exploiting changes in people's long-term and momentary contexts respectively. While the intervention-selection model adapts the intervention delivery with respect to type and frequency, the opportune-moment-identification model tries to find the most opportune moments to deliver interventions throughout a day. We propose two accelerator techniques over the standard reinforcement learning algorithms to boost learning performance. First, we propose a customized version of eligibility traces for rewarding past actions throughout an agent's trajectory. Second, we utilize the transfer learning method to reuse knowledge across multiple learning environments. We validate the proposed approach in a simulated experiment where we simulate four personas differing in their daily activities, preferences on specific intervention types and attitudes towards the targeted behavior. Our experiments show that the proposed approach yields better results compared to the standard reinforcement learning algorithms and successfully capture the simulated variations associated with the personas.

An expandable approach for design and personalization of digital, just-in-time adaptive interventions.

Objective: We aim to deliver a framework with 2 main objectives: 1) facilitating the design of theory-driven, adaptive, digital interventions addressing chronic illnesses or health problems and 2) producing personalized intervention delivery strategies to support self-management by optimizing various intervention components tailored to people's individual needs, momentary contexts, and psychosocial variables. Materials and Methods: We propose a template-based digital intervention design mechanism enabling the configuration of evidence-based, just-in-time, adaptive intervention components. The design mechanism incorporates a rule definition language enabling experts to specify triggering conditions for interventions based on momentary and historical contextual/personal data. The framework continuously monitors and processes personal data space and evaluates intervention-triggering conditions. We benefit from reinforcement learning methods to develop personalized intervention delivery strategies with respect to timing, frequency, and type (content) of interventions. To validate the personalization algorithm, we lay out a simulation testbed with 2 personas, differing in their various simulated real-life conditions. Results: We evaluate the design mechanism by presenting example intervention definitions based on behavior change taxonomies and clinical guidelines. Furthermore, we provide intervention definitions for a real-world care program targeting diabetes patients. Finally, we validate the personalized delivery mechanism through a set of hypotheses, asserting certain ways of adaptation in the delivery strategy, according to the differences in simulation related to personal preferences, traits, and lifestyle patterns. Conclusion: While the design mechanism is sufficiently expandable to meet the theoretical and clinical intervention design requirements, the personalization algorithm is capable of adapting intervention delivery strategies for simulated real-life conditions.