Understanding the impact of an AI-enabled conversational agent mobile app on users' mental health and wellbeing with a self-reported maternal event: a mixed method real-world data mHealth study.

Background: Maternal mental health care is variable and with limited accessibility. Artificial intelligence (AI) conversational agents (CAs) could potentially play an important role in supporting maternal mental health and wellbeing. Our study examined data from real-world users who self-reported a maternal event while engaging with a digital mental health and wellbeing AI-enabled CA app (Wysa) for emotional support. The study evaluated app effectiveness by comparing changes in self-reported depressive symptoms between a higher engaged group of users and a lower engaged group of users and derived qualitative insights into the behaviors exhibited among higher engaged maternal event users based on their conversations with the AI CA. Methods: Real-world anonymised data from users who reported going through a maternal event during their conversation with the app was analyzed. For the first objective, users who completed two PHQ-9 self-reported assessments (n = 51) were grouped as either higher engaged users (n = 28) or lower engaged users (n = 23) based on their number of active session-days with the CA between two screenings. A non-parametric Mann-Whitney test (M-W) and non-parametric Common Language effect size was used to evaluate group differences in self-reported depressive symptoms. For the second objective, a Braun and Clarke thematic analysis was used to identify engagement behavior with the CA for the top quartile of higher engaged users (n = 10 of 51). Feedback on the app and demographic information was also explored. Results: Results revealed a significant reduction in self-reported depressive symptoms among the higher engaged user group compared to lower engaged user group (M-W p = .004) with a high effect size (CL = 0.736). Furthermore, the top themes that emerged from the qualitative analysis revealed users expressed concerns, hopes, need for support, reframing their thoughts and expressing their victories and gratitude. Conclusion: These findings provide preliminary evidence of the effectiveness and engagement and comfort of using this AI-based emotionally intelligent mobile app to support mental health and wellbeing across a range of maternal events and experiences.

Customizable Natural Language Processing Biomarker Extraction Tool.

PURPOSE: Natural language processing (NLP) in pathology reports to extract biomarker information is an ongoing area of research. MetaMap is a natural language processing tool developed and funded by the National Library of Medicine to map biomedical text to the Unified Medical Language System Metathesaurus by applying specific tags to clinically relevant terms. Although results are useful without additional postprocessing, these tags lack important contextual information. METHODS: Our novel method takes terminology-driven semantic tags and incorporates those into a semantic frame that is task-specific to add necessary context to MetaMap. We use important contextual information to capture biomarker results to support Community Health System's use of Precision Medicine treatments for patients with cancer. For each biomarker, the name, type, numeric quantifiers, non-numeric qualifiers, and the time frame are extracted. These fields then associate biomarkers with their context in the pathology report such as test type, probe intensity, copy-number changes, and even failed results. A selection of 6,713 relevant reports contained the following standard-of-care biomarkers for metastatic breast cancer: breast cancer gene 1 and 2, estrogen receptor, progesterone receptor, human epidermal growth factor receptor 2, and programmed death-ligand 1. RESULTS: The method was tested on pathology reports from the internal pathology laboratory at Henry Ford Health System. A certified tumor registrar reviewed 400 tests, which showed > 95% accuracy for all extracted biomarker types. CONCLUSION: Using this new method, it is possible to extract high-quality, contextual biomarker information, and this represents a significant advance in biomarker extraction.

Validation of a Mortality Composite Score in the Real-World Setting: Overcoming Source-Specific Disparities and Biases.

PURPOSE: This study tested whether a composite mortality score could overcome gaps and potential biases in individual real-world mortality data sources. Complete and accurate mortality data are necessary to calculate important outcomes in oncology, including overall survival. However, in the United States, there is not a single complete and broadly applicable mortality data source. It is further likely that available data sources are biased in their coverage of sex, race, age, and socioeconomic status (SES). METHODS: Six individual real-world data sources were combined to develop a high-quality composite mortality score. The composite score was benchmarked against the gold standard for mortality data, the National Death Index. Subgroup analyses were then conducted to evaluate the completeness and accuracy by sex, race, age, and SES. RESULTS: The composite mortality score achieved a sensitivity of 94.9% and specificity of 92.8% compared with the National Death Index, with concordance within 1 day of 98.6%. Although some individual data sources show significant coverage gaps related to sex, race, age, and SES, the composite score maintains high sensitivity (84.6%-96.1%) and specificity (77.9%-99.2%) across subgroups. CONCLUSION: A composite score leveraging multiple scalable sources for mortality in the real-world setting maintained strong sensitivity, specificity, and concordance, including across sex, race, age, and SES subgroups.

Digital Health Management During and Beyond the COVID-19 Pandemic: Opportunities, Barriers, and Recommendations.

During the coronavirus disease (COVID-19) crisis, digital technologies have become a major route for accessing remote care. Therefore, the need to ensure that these tools are safe and effective has never been greater. We raise five calls to action to ensure the safety, availability, and long-term sustainability of these technologies: (1) due diligence: remove harmful health apps from app stores; (2) data insights: use relevant health data insights from high-quality digital tools to inform the greater response to COVID-19; (3) freely available resources: make high-quality digital health tools available without charge, where possible, and for as long as possible, especially to those who are most vulnerable; (4) digital transitioning: transform conventional offline mental health services to make them digitally available; and (5) population self-management: encourage governments and insurers to work with developers to look at how digital health management could be subsidized or funded. We believe this should be carried out at the population level, rather than at a prescription level.

Using Passive Smartphone Sensing for Improved Risk Stratification of Patients With Depression and Diabetes: Cross-Sectional Observational Study.

BACKGROUND: Research studies are establishing the use of smartphone sensing to measure mental well-being. Smartphone sensor information captures behavioral patterns, and its analysis helps reveal well-being changes. Depression in diabetes goes highly underdiagnosed and underreported. The comorbidity has been associated with increased mortality and worse clinical outcomes, including poor glycemic control and self-management. Clinical-only intervention has been found to have a very modest effect on diabetes management among people with depression. Smartphone technologies could play a significant role in complementing comorbid care. OBJECTIVE: This study aimed to analyze the association between smartphone-sensing parameters and symptoms of depression and to explore an approach to risk-stratify people with diabetes. METHODS: A cross-sectional observational study (Project SHADO-Analyzing Social and Health Attributes through Daily Digital Observation) was conducted on 47 participants with diabetes. The study's smartphone-sensing app passively collected data regarding activity, mobility, sleep, and communication from each participant. Self-reported symptoms of depression using a validated Patient Health Questionnaire-9 (PHQ-9) were collected once every 2 weeks from all participants. A descriptive analysis was performed to understand the representation of the participants. A univariate analysis was performed on each derived sensing variable to compare behavioral changes between depression states-those with self-reported major depression (PHQ-9>9) and those with none (PHQ-9≤9). A classification predictive modeling, using supervised machine-learning methods, was explored using derived sensing variables as input to construct and compare classifiers that could risk-stratify people with diabetes based on symptoms of depression. RESULTS: A noticeably high prevalence of self-reported depression (30 out of 47 participants, 63%) was found among the participants. Between depression states, a significant difference was found for average activity rates (daytime) between participant-day instances with symptoms of major depression (mean 16.06 [SD 14.90]) and those with none (mean 18.79 [SD 16.72]), P=.005. For average number of people called (calls made and received), a significant difference was found between participant-day instances with symptoms of major depression (mean 5.08 [SD 3.83]) and those with none (mean 8.59 [SD 7.05]), P<.001. These results suggest that participants with diabetes and symptoms of major depression exhibited lower activity through the day and maintained contact with fewer people. Using all the derived sensing variables, the extreme gradient boosting machine-learning classifier provided the best performance with an average cross-validation accuracy of 79.07% (95% CI 74%-84%) and test accuracy of 81.05% to classify symptoms of depression. CONCLUSIONS: Participants with diabetes and self-reported symptoms of major depression were observed to show lower levels of social contact and lower activity levels during the day. Although findings must be reproduced in a broader randomized controlled study, this study shows promise in the use of predictive modeling for early detection of symptoms of depression in people with diabetes using smartphone-sensing information.

An Empathy-Driven, Conversational Artificial Intelligence Agent (Wysa) for Digital Mental Well-Being: Real-World Data Evaluation Mixed-Methods Study.

BACKGROUND: A World Health Organization 2017 report stated that major depression affects almost 5% of the human population. Major depression is associated with impaired psychosocial functioning and reduced quality of life. Challenges such as shortage of mental health personnel, long waiting times, perceived stigma, and lower government spends pose barriers to the alleviation of mental health problems. Face-to-face psychotherapy alone provides only point-in-time support and cannot scale quickly enough to address this growing global public health challenge. Artificial intelligence (AI)-enabled, empathetic, and evidence-driven conversational mobile app technologies could play an active role in filling this gap by increasing adoption and enabling reach. Although such a technology can help manage these barriers, they should never replace time with a health care professional for more severe mental health problems. However, app technologies could act as a supplementary or intermediate support system. Mobile mental well-being apps need to uphold privacy and foster both short- and long-term positive outcomes. OBJECTIVE: This study aimed to present a preliminary real-world data evaluation of the effectiveness and engagement levels of an AI-enabled, empathetic, text-based conversational mobile mental well-being app, Wysa, on users with self-reported symptoms of depression. METHODS: In the study, a group of anonymous global users were observed who voluntarily installed the Wysa app, engaged in text-based messaging, and self-reported symptoms of depression using the Patient Health Questionnaire-9. On the basis of the extent of app usage on and between 2 consecutive screening time points, 2 distinct groups of users (high users and low users) emerged. The study used mixed-methods approach to evaluate the impact and engagement levels among these users. The quantitative analysis measured the app impact by comparing the average improvement in symptoms of depression between high and low users. The qualitative analysis measured the app engagement and experience by analyzing in-app user feedback and evaluated the performance of a machine learning classifier to detect user objections during conversations. RESULTS: The average mood improvement (ie, difference in pre- and post-self-reported depression scores) between the groups (ie, high vs low users; n=108 and n=21, respectively) revealed that the high users group had significantly higher average improvement (mean 5.84 [SD 6.66]) compared with the low users group (mean 3.52 [SD 6.15]); Mann-Whitney P=.03 and with a moderate effect size of 0.63. Moreover, 67.7% of user-provided feedback responses found the app experience helpful and encouraging. CONCLUSIONS: The real-world data evaluation findings on the effectiveness and engagement levels of Wysa app on users with self-reported symptoms of depression show promise. However, further work is required to validate these initial findings in much larger samples and across longer periods.