Proactive Identification of Patients with Diabetes at Risk of Uncontrolled Outcomes during a Diabetes Management Program: Conceptualization and Development Study Using Machine Learning.

BACKGROUND: The growth in the capabilities of telehealth have made it possible to identify individuals with a higher risk of uncontrolled diabetes and provide them with targeted support and resources to help them manage their condition. Thus, predictive modeling has emerged as a valuable tool for the advancement of diabetes management. OBJECTIVE: This study aimed to conceptualize and develop a novel machine learning (ML) approach to proactively identify participants enrolled in a remote diabetes monitoring program (RDMP) who were at risk of uncontrolled diabetes at 12 months in the program. METHODS: Registry data from the Livongo for Diabetes RDMP were used to design separate dynamic predictive ML models to predict participant outcomes at each monthly checkpoint of the participants' program journey (month-n models) from the first day of onboarding (month-0 model) up to the 11th month (month-11 model). A participant's program journey began upon onboarding into the RDMP and monitoring their own blood glucose (BG) levels through the RDMP-provided BG meter. Each participant passed through 12 predicative models through their first year enrolled in the RDMP. Four categories of participant attributes (ie, survey data, BG data, medication fills, and health signals) were used for feature construction. The models were trained using the light gradient boosting machine and underwent hyperparameter tuning. The performance of the models was evaluated using standard metrics, including precision, recall, specificity, the area under the curve, the F1-score, and accuracy. RESULTS: The ML models exhibited strong performance, accurately identifying observable at-risk participants, with recall ranging from 70% to 94% and precision from 40% to 88% across the 12-month program journey. Unobservable at-risk participants also showed promising performance, with recall ranging from 61% to 82% and precision from 42% to 61%. Overall, model performance improved as participants progressed through their program journey, demonstrating the importance of engagement data in predicting long-term clinical outcomes. CONCLUSIONS: This study explored the Livongo for Diabetes RDMP participants' temporal and static attributes, identification of diabetes management patterns and characteristics, and their relationship to predict diabetes management outcomes. Proactive targeting ML models accurately identified participants at risk of uncontrolled diabetes with a high level of precision that was generalizable through future years within the RDMP. The ability to identify participants who are at risk at various time points throughout the program journey allows for personalized interventions to improve outcomes. This approach offers significant advancements in the feasibility of large-scale implementation in remote monitoring programs and can help prevent uncontrolled glycemic levels and diabetes-related complications. Future research should include the impact of significant changes that can affect a participant's diabetes management.

The Effect of a Digital Mental Health Program on Anxiety and Depression Symptoms: Retrospective Analysis of Clinical Severity.

BACKGROUND: Evidence-based digital health programs have shown efficacy in being primary tools to improve emotional and mental health, as well as offering supplementary support to individuals undergoing psychotherapy for anxiety, depression, and other mental health disorders. However, information is lacking about the dose response to digital mental health interventions. OBJECTIVE: The objective of the study was to examine the effect of time in program and program usage on symptom change among individuals enrolled in a real-world comprehensive digital mental health program (myStrength) who are experiencing severe anxiety or depression. METHODS: Eligible participants (N=18,626) were adults aged 18 years and older who were enrolled in myStrength for at least four weeks as part of their employee wellness benefit program, who completed baseline, the 2-week, 2-month, and 6-month surveys querying symptoms of anxiety (Generalized Anxiety Disorder-7 [GAD-7]) and depression (Patient Health Questionnaire-9 [PHQ-9]). Linear growth curve models were used to analyze the effect of average weekly program usage on subsequent GAD-7 and PHQ-9 scores for participants with scores indicating severe anxiety (GAD-7≥15) or depression (PHQ-9≥15). All models were adjusted for baseline score and demographics. RESULTS: Participants in the study (N=1519) were 77.4% female (1176/1519), had a mean age of 45 years (SD 14 years), and had an average enrollment time of 3 months. At baseline, participants reported an average of 9.39 (SD 6.04) on the GAD-7 and 11.0 (SD 6.6) on the PHQ-9. Those who reported 6-month results had an average of 8.18 (SD 6.15) on the GAD-7 and 9.18 (SD 6.79) on the PHQ-9. Participants with severe scores (n=506) experienced a significant improvement of 2.97 (SE 0.35) and 3.97 (SE 0.46) at each time point for anxiety and depression, respectively (t=-8.53 and t=-8.69, respectively; Ps<.001). Those with severe baseline scores also saw a reduction of 0.27 (SE 0.08) and 0.25 (SE 0.09) points in anxiety and depression, respectively, for each additional program activity per week (t=-3.47 and t=-2.66, respectively; Ps<.05). CONCLUSIONS: For participants with severe baseline scores, the study found a clinically significant reduction of approximately 9 points for anxiety and 12 points for depression after 6 months of enrollment, suggesting that interventions targeting mental health must maintain active, ongoing engagement when symptoms are present and be available as a continuous resource to maximize clinical impact, specifically in those experiencing severe anxiety or depression. Moreover, a dosing effect was shown, indicating improvement in outcomes among participants who engaged with the program every other day for both anxiety and depression. This suggests that digital mental health programs that provide both interesting and evidence-based activities could be more successful in further improving mental health outcomes.

Improving Outcomes Through Personalized Recommendations in a Remote Diabetes Monitoring Program: Observational Study.

BACKGROUND: Diabetes management is complex, and program personalization has been identified to enhance engagement and clinical outcomes in diabetes management programs. However, 50% of individuals living with diabetes are unable to achieve glycemic control, presenting a gap in the delivery of self-management education and behavior change. Machine learning and recommender systems, which have been used within the health care setting, could be a feasible application for diabetes management programs to provide a personalized user experience and improve user engagement and outcomes. OBJECTIVE: This study aims to evaluate machine learning models using member-level engagements to predict improvement in estimated A1c and develop personalized action recommendations within a remote diabetes monitoring program to improve clinical outcomes. METHODS: A retrospective study of Livongo for Diabetes member engagement data was analyzed within five action categories (interacting with a coach, reading education content, self-monitoring blood glucose level, tracking physical activity, and monitoring nutrition) to build a member-level model to predict if a specific type and level of engagement could lead to improved estimated A1c for members with type 2 diabetes. Engagement and improvement in estimated A1c can be correlated; therefore, the doubly robust learning method was used to model the heterogeneous treatment effect of action engagement on improvements in estimated A1c. RESULTS: The treatment effect was successfully computed within the five action categories on estimated A1c reduction for each member. Results show interaction with coaches and self-monitoring blood glucose levels were the actions that resulted in the highest average decrease in estimated A1c (1.7% and 1.4%, respectively) and were the most recommended actions for 54% of the population. However, these were found to not be the optimal interventions for all members; 46% of members were predicted to have better outcomes with one of the other three interventions. Members who engaged with their recommended actions had on average a 0.8% larger reduction in estimated A1c than those who did not engage in recommended actions within the first 3 months of the program. CONCLUSIONS: Personalized action recommendations using heterogeneous treatment effects to compute the impact of member actions can reduce estimated A1c and be a valuable tool for diabetes management programs in encouraging members toward actions to improve clinical outcomes.

Association of physical activity on blood glucose in individuals with type 2 diabetes.

Regular physical activity (PA) has been shown to improve glycemic control in persons with type 2 diabetes. This study aimed to investigate the impact of PA on blood glucose after controlling for medication use, demographics, and week of activation using a real-world population of individuals with type 2 diabetes. A longitudinal, retrospective study was performed evaluating weekly PA of Livongo members (N = 9,509), which analyzed fasting blood glucose (FBG), step counts, and daily active minutes. Linear mixed-effect modeling technique was used to investigate within member and between member effects of input variables on average weekly FBG. Of members enrolled, 6,336 (32%) had self-reported body mass index, qualified week with diabetes medications, and FBG measures. Members' baseline average age was 49.4 (SD 10.1) years old, 43% female, and 45,496 member weeks with an average of 7.2 qualified weeks (PA observable in ≥4 days) per member. Average weekly FBG was 140.5 mg/dL (SD 39.8), and average daily step counts were 4,833 (SD 3,266). Moving from sedentary (<5,000 steps per day) to active (≥5,000 steps per day) resulted in mean weekly FBG reduction of 13 mg/dL (95% CI: -22.6 to -3.14). One additional day of ≥8,000 steps reduced mean weekly FBG by 0.47 mg/dL (95% CI: -0.77 to -0.16). Members who completed 30 min of moderate to vigorous PA above the population average reduced mean weekly FBG by 7.7 mg/dL (95% CI: -13.4 to -2.0). PA is associated with a mean weekly FBG reduction of 13 mg/dL when changing from a sedentary to active lifestyle while participating in a remote diabetes monitoring program.

Figure Correction: The Effect of a Cellular-Enabled Glucose Meter on Glucose Control for Patients With Diabetes: Prospective Pre-Post Study.

[This corrects the article DOI: 10.2196/14799.].

Drivers of weight loss in a CDC-recognized digital diabetes prevention program.

INTRODUCTION: To investigate the impact of the digital Livongo Diabetes Prevention Program (DPP) on weight at 12 months, understand participants' self-monitoring behaviors associated with greater weight loss, and evaluate the impact of coaching interactions on more frequent self-monitoring behaviors. RESEARCH DESIGN AND METHODS: A retrospective analysis was performed using data from 2037 participants enrolled in the Livongo DPP who completed lesson 1 and recorded a starting weight during 2016-2017. Self-monitoring behaviors, including weigh-ins, food logging, activity, and coach-participant interactions, were analyzed at 6 and 12 months. Subgroup analysis was conducted based on those who were highly engaged versus those minimally engaged. Multiple regression analysis was performed using demographic, self-monitoring, and lesson attendance data to determine predictors of weight loss at 12 months and coaching impact on self-monitoring. RESULTS: Participants had a mean age of 50 years (SD ±12), with a starting weight of 94 kg (SD ±21), were college-educated (78%), and were female (74%). Overall, participants lost on average 5.1% of their starting weight. Highly engaged participants lost 6.6% of starting body weight, with 25% losing ≥10% at 12 months. Logistic regression analysis showed each submitted food log was associated with 0.23 kg (p<0.05) weight loss, each lesson completed was associated with 0.14 kg (p<0.05) weight loss, and a week of 150 active minutes was associated with 0.1 kg (p<0.01) weight loss. One additional coach-participant message each week was associated with 1.4 more food logs per week, 1.6% increase in weeks with four or more weigh-ins, and a 2.7% increase in weeks with 150 min of activity. CONCLUSIONS: Food logging had the largest impact on weight loss, followed by lesson engagement and physical activity. Future studies should examine further opportunities to deliver nutrition-based content to increase and sustain weight loss for DPP.

The Effect of a Cellular-Enabled Glucose Meter on Glucose Control for Patients With Diabetes: Prospective Pre-Post Study.

BACKGROUND: Diabetes is a global epidemic affecting approximately 30 million people in the United States. The World Health Organization recommends using technology and telecommunications to improve health care delivery and disease management. The Livongo for Diabetes Program offers a remote monitoring technology with Certified Diabetes Educator outreach. OBJECTIVE: The purpose of this study was to examine health outcomes measured by changes in HbA1c, in time in target blood glucose range, and in depression symptoms for patients enrolled in a remote digital diabetes management program in a Diabetes Center of Excellence setting. METHODS: The impact of the Livongo for Diabetes program on hemoglobin A1c (HbA1c), blood glucose ranges, and depression screening survey results (Patient Health Questionnaire-2 [PHQ-2]) were assessed over 12 months in a prospective cohort recruited from the University of South Florida Health Diabetes Home for Healthy Living. Any patient ≥18 years old with a diagnosis of diabetes was approached for voluntary inclusion into the program. The analysis was a pre-post design for those members enrolled in the study. Data was collected at outpatient clinic visits and remotely through the Livongo glucose meter. RESULTS: A total of 86 adults were enrolled into the Livongo for Diabetes program, with 49% (42/86) female, an average age of 50 (SD 15) years, 56% (48/86) with type 2 diabetes mellitus, and 69% (59/86) with insulin use. The mean HbA1c drop amongst the group was 0.66% (P=.17), with all participants showing a decline in HbA1c at 12 months. A 17% decrease of blood glucose checks <70 mg/dL occurred concurrently. Participants with type 2 diabetes not using insulin had blood glucose values within target range (70-180 mg/dL) 89% of the time. Participants with type 2 diabetes using insulin were in target range 68% of the time, and type 1 diabetes 58% of the time. Average PHQ-2 scores decreased by 0.56 points during the study period. CONCLUSIONS: Participants provided with a cellular-enabled blood glucose meter with real-time feedback and access to coaching from a certified diabetes educator in an outpatient clinical setting experienced improved mean glucose values and fewer episodes of hypoglycemia relative to the start of the program.

Reduced medical spending associated with increased use of a remote diabetes management program and lower mean blood glucose values.

Aims: Many new mobile technologies are available to assist people in managing chronic conditions, but data on the association between the use of these technologies and medical spending remains limited. As the available digital technology offerings to aid in diabetes management increase, it is important to understand their impact on medical spending. The aim of this study was to investigate the financial impact of a remote digital diabetes management program using medical claims and real-time blood glucose data. Materials and methods: A retrospective analysis of multivariate difference-in-difference and instrumental variables regression modeling was performed using data collected from a remote digital diabetes management program. All employees with diabetes were invited, in a phased introduction, to join the program. Data included blood glucose (BG) values captured remotely from members via connected BG meters and medical spending claims. Participants included members (those who accepted the invitation, n = 2,261) and non-members (n = 8,741) who received health insurance benefits from three self-insured employers. Medical spending was compared between people with well-controlled (BG ≤ 154 mg/dL) and poorly controlled (BG > 154 mg/dL) diabetes. Results: Program access was associated with a 21.9% (p < 0.01) decrease in medical spending, which translates into a $88 saving per member per month at 1 year. Compared to non-members, members experienced a 10.7% (p < 0.01) reduction in diabetes-related medical spending and a 24.6% (p < 0.01) reduction in spending on office-based services. Well-controlled BG values were associated with 21.4% (p = 0.03) lower medical spending. Limitations and conclusions: Remote digital diabetes management is associated with decreased medical spending at 1 year. Reductions in spending increased with active utilization. It will be beneficial for future studies to analyze the long-term effects of the remote diabetes management program and assess impacts on patient health and well-being.

Expert Coaching in Weight Loss: Retrospective Analysis.

BACKGROUND: Providing coaches as part of a weight management program is a common practice to increase participant engagement and weight loss success. Understanding coach and participant interactions and how these interactions impact weight loss success needs to be further explored for coaching best practices. OBJECTIVE: The purpose of this study was to analyze the coach and participant interaction in a 6-month weight loss intervention administered by Retrofit, a personalized weight management and Web-based disease prevention solution. The study specifically examined the association between different methods of coach-participant interaction and weight loss and tried to understand the level of coaching impact on weight loss outcome. METHODS: A retrospective analysis was performed using 1432 participants enrolled from 2011 to 2016 in the Retrofit weight loss program. Participants were males and females aged 18 years or older with a baseline body mass index of ≥25 kg/m², who also provided at least one weight measurement beyond baseline. First, a detailed analysis of different coach-participant interaction was performed using both intent-to-treat and completer populations. Next, a multiple regression analysis was performed using all measures associated with coach-participant interactions involving expert coaching sessions, live weekly expert-led Web-based classes, and electronic messaging and feedback. Finally, 3 significant predictors (P<.001) were analyzed in depth to reveal the impact on weight loss outcome. RESULTS: Participants in the Retrofit weight loss program lost a mean 5.14% (SE 0.14) of their baseline weight, with 44% (SE 0.01) of participants losing at least 5% of their baseline weight. Multiple regression model (R2=.158, P<.001) identified the following top 3 measures as significant predictors of weight loss at 6 months: expert coaching session attendance (P<.001), live weekly Web-based class attendance (P<.001), and food log feedback days per week (P<.001). Attending 80% of expert coaching sessions, attending 60% of live weekly Web-based classes, and receiving a minimum of 1 food log feedback day per week were associated with clinically significant weight loss. CONCLUSIONS: Participant's one-on-one expert coaching session attendance, live weekly expert-led interactive Web-based class attendance, and the number of food log feedback days per week from expert coach were significant predictors of weight loss in a 6-month intervention.

What Matters in Weight Loss? An In-Depth Analysis of Self-Monitoring.

BACKGROUND: Using technology to self-monitor body weight, dietary intake, and physical activity is a common practice used by consumers and health companies to increase awareness of current and desired behaviors in weight loss. Understanding how to best use the information gathered by these relatively new methods needs to be further explored. OBJECTIVE: The purpose of this study was to analyze the contribution of self-monitoring to weight loss in participants in a 6-month commercial weight-loss intervention administered by Retrofit and to specifically identify the significant contributors to weight loss that are associated with behavior and outcomes. METHODS: A retrospective analysis was performed using 2113 participants enrolled from 2011 to 2015 in a Retrofit weight-loss program. Participants were males and females aged 18 years or older with a starting body mass index of ≥25 kg/m2, who also provided a weight measurement at the sixth month of the program. Multiple regression analysis was performed using all measures of self-monitoring behaviors involving weight measurements, dietary intake, and physical activity to predict weight loss at 6 months. Each significant predictor was analyzed in depth to reveal the impact on outcome. RESULTS: Participants in the Retrofit Program lost a mean -5.58% (SE 0.12) of their baseline weight with 51.87% (1096/2113) of participants losing at least 5% of their baseline weight. Multiple regression model (R2=.197, P<0.001) identified the following measures as significant predictors of weight loss at 6 months: number of weigh-ins per week (P<.001), number of steps per day (P=.02), highly active minutes per week (P<.001), number of food log days per week (P<.001), and the percentage of weeks with five or more food logs (P<.001). Weighing in at least three times per week, having a minimum of 60 highly active minutes per week, food logging at least three days per week, and having 64% (16.6/26) or more weeks with at least five food logs were associated with clinically significant weight loss for both male and female participants. CONCLUSIONS: The self-monitoring behaviors of self-weigh-in, daily steps, high-intensity activity, and persistent food logging were significant predictors of weight loss during a 6-month intervention.

Retrofit Weight-Loss Outcomes at 6, 12, and 24 Months and Characteristics of 12-Month High Performers: A Retrospective Analysis.

BACKGROUND: Obesity is the leading cause of preventable death costing the health care system billions of dollars. Combining self-monitoring technology with personalized behavior change strategies results in clinically significant weight loss. However, there is a lack of real-world outcomes in commercial weight-loss program research. OBJECTIVE: Retrofit is a personalized weight management and disease-prevention solution. This study aimed to report Retrofit's weight-loss outcomes at 6, 12, and 24 months and characterize behaviors, age, and sex of high-performing participants who achieved weight loss of 10% or greater at 12 months. METHODS: A retrospective analysis was performed from 2011 to 2014 using 2720 participants enrolled in a Retrofit weight-loss program. Participants had a starting body mass index (BMI) of >25 kg/m² and were at least 18 years of age. Weight measurements were assessed at 6, 12, and 24 months in the program to evaluate change in body weight, BMI, and percentage of participants who achieved 5% or greater weight loss. A secondary analysis characterized high-performing participants who lost ≥10% of their starting weight (n=238). Characterized behaviors were evaluated, including self-monitoring through weigh-ins, number of days wearing an activity tracker, daily step count average, and engagement through coaching conversations via Web-based messages, and number of coaching sessions attended. RESULTS: Average weight loss at 6 months was -5.55% for male and -4.86% for female participants. Male and female participants had an average weight loss of -6.28% and -5.37% at 12 months, respectively. Average weight loss at 24 months was -5.03% and -3.15% for males and females, respectively. Behaviors of high-performing participants were assessed at 12 months. Number of weigh-ins were greater in high-performing male (197.3 times vs 165.4 times, P=.001) and female participants (222 times vs 167 times, P<.001) compared with remaining participants. Total activity tracker days and average steps per day were greater in high-performing females (304.7 vs 266.6 days, P<.001; 8380.9 vs 7059.7 steps, P<.001, respectively) and males (297.1 vs 255.3 days, P<.001; 9099.3 vs 8251.4 steps, P=.008, respectively). High-performing female participants had significantly more coaching conversations via Web-based messages than remaining female participants (341.4 vs 301.1, P=.03), as well as more days with at least one such electronic message (118 vs 108 days, P=.03). High-performing male participants displayed similar behavior. CONCLUSIONS: Participants on the Retrofit program lost an average of -5.21% at 6 months, -5.83% at 12 months, and -4.09% at 24 months. High-performing participants show greater adherence to self-monitoring behaviors of weighing in, number of days wearing an activity tracker, and average number of steps per day. Female high performers have higher coaching engagement through conversation days and total number of coaching conversations.