Bridging the Skills Gap: Evaluating an AI-Assisted Provider Platform to Support Care Providers with Empathetic Delivery of Protocolized Therapy.

Despite the high prevalence and burden of mental health conditions, there is a global shortage of mental health providers. Artificial Intelligence (AI) methods have been proposed as a way to address this shortage, by supporting providers with less extensive training as they deliver care. To this end, we developed the AI-Assisted Provider Platform (A2P2), a text-based virtual therapy interface that includes a response suggestion feature, which supports providers in delivering protocolized therapies empathetically. We studied providers with and without expertise in mental health treatment delivering a therapy session using the platform with (intervention) and without (control) AI-assistance features. Upon evaluation, the AI-assisted system significantly decreased response times by 29.34% (p=0.002), tripled empathic response accuracy (p=0.0001), and increased goal recommendation accuracy by 66.67% (p=0.001) across both user groups compared to the control. Both groups rated the system as having excellent usability.

Understanding User Experience: Exploring Participants' Messages With a Web-Based Behavioral Health Intervention for Adolescents With Chronic Pain.

BACKGROUND: Delivery of behavioral health interventions on the internet offers many benefits, including accessibility, cost-effectiveness, convenience, and anonymity. In recent years, an increased number of internet interventions have been developed, targeting a range of conditions and behaviors, including depression, pain, anxiety, sleep disturbance, and eating disorders. Human support (coaching) is a common component of internet interventions that is intended to boost engagement; however, little is known about how participants interact with coaches and how this may relate to their experience with the intervention. By examining the data that participants produce during an intervention, we can characterize their interaction patterns and refine treatments to address different needs. OBJECTIVE: In this study, we employed text mining and visual analytics techniques to analyze messages exchanged between coaches and participants in an internet-delivered pain management intervention for adolescents with chronic pain and their parents. METHODS: We explored the main themes in coaches' and participants' messages using an automated textual analysis method, topic modeling. We then clustered participants' messages to identify subgroups of participants with similar engagement patterns. RESULTS: First, we performed topic modeling on coaches' messages. The themes in coaches' messages fell into 3 categories: Treatment Content, Administrative and Technical, and Rapport Building. Next, we employed topic modeling to identify topics from participants' message histories. Similar to the coaches' topics, these were subsumed under 3 high-level categories: Health Management and Treatment Content, Questions and Concerns, and Activities and Interests. Finally, the cluster analysis identified 4 clusters, each with a distinguishing characteristic: Assignment-Focused, Short Message Histories, Pain-Focused, and Activity-Focused. The name of each cluster exemplifies the main engagement patterns of that cluster. CONCLUSIONS: In this secondary data analysis, we demonstrated how automated text analysis techniques could be used to identify messages of interest, such as questions and concerns from users. In addition, we demonstrated how cluster analysis could be used to identify subgroups of individuals who share communication and engagement patterns, and in turn facilitate personalization of interventions for different subgroups of patients. This work makes 2 key methodological contributions. First, this study is innovative in its use of topic modeling to provide a rich characterization of the textual content produced by coaches and participants in an internet-delivered behavioral health intervention. Second, to our knowledge, this is the first example of the use of a visual analysis method to cluster participants and identify similar patterns of behavior based on intervention message content.

A Systematic Review of Health Dialog Systems.

BACKGROUND: Health dialog systems have seen increased adoption by patients, hospitals, and universities due to the confluence of advancements in machine learning and the ubiquity of high-performance hardware that supports real-time speech recognition, high-fidelity text-to-speech, and semantic understanding of natural language. OBJECTIVES: This review seeks to enumerate opportunities to apply dialog systems toward the improvement of health outcomes while identifying both gaps in the current literature that may impede their implementation and recommendations that may improve their success in medical practice. METHODS: A search over PubMed and the ACM Digital Library was conducted on September 12, 2017 to collect all articles related to dialog systems within the domain of health care. These results were screened for eligibility with the main criteria being a peer-reviewed study of a system that includes both a natural language interface and either end-user testing or practical implementation. RESULTS: Forty-six studies met the inclusion criteria including 24 quasi-experimental studies, 16 randomized control trials, 2 case-control studies, 2 prospective cohort studies, 1 system description, and 1 human-computer conversation analysis. These studies evaluated dialog systems in five application domains: medical education (n = 20), clinical processes (n = 14), mental health (n = 5), personal health agents (n = 5), and patient education (n = 2). CONCLUSION: We found that dialog systems have been widely applied to health care; however, most studies are not reproducible making direct comparison between systems and independent confirmation of findings difficult. Widespread adoption will also require the adoption of standard evaluation and reporting methods for health dialog systems to demonstrate clinical significance.

Resource and Response Type Classification for Consumer Health Question Answering.

Health question answering systems often depend on the initial step of question type classification. Practitioners face several modeling choices for this component alone. We evaluate the effectiveness of different modeling choices in both the embeddings and architectural hyper-parameters of the classifier. In the process, we achieve improved performance over previous methods, achieving a new best 5-fold accuracy of 85.3% on the GARD dataset. The contribution of this work is to evaluate the performance of sentence classification methods on the task of consumer health question type classification and to contribute a dataset of 2,882 medical questions annotated for question type.