Usability Issues in Evidence-Based Psychosocial Interventions and Implementation Strategies: Cross-project Analysis.

BACKGROUND: People often prefer evidence-based psychosocial interventions (EBPIs) for mental health care; however, these interventions frequently remain unavailable to people in nonspecialty or integrated settings, such as primary care and schools. Previous research has suggested that usability, a concept from human-centered design, could support an understanding of the barriers to and facilitators of the successful adoption of EBPIs and support the redesign of EBPIs and implementation strategies. OBJECTIVE: This study aimed to identify and categorize usability issues in EBPIs and their implementation strategies. METHODS: We adapted a usability issue analysis and reporting format from a human-centered design. A total of 13 projects supported by the National Institute of Mental Health-funded Accelerating the Reach and Impact of Treatments for Youth and Adults with Mental Illness Center at the University of Washington used this format to describe usability issues for EBPIs and implementation strategies with which they were working. Center researchers used iterative affinity diagramming and coding processes to identify usability issue categories. On the basis of these categories and the underlying issues, we propose heuristics for the design or redesign of EBPIs and implementation strategies. RESULTS: The 13 projects reported a total of 90 usability issues, which we categorized into 12 categories, including complex and/or cognitively overwhelming, required time exceeding available time, incompatibility with interventionist preference or practice, incompatibility with existing workflow, insufficient customization to clients/recipients, intervention buy-in (value), interventionist buy-in (trust), overreliance on technology, requires unavailable infrastructure, inadequate scaffolding for client/recipient, inadequate training and scaffolding for interventionists, and lack of support for necessary communication. These issues range from minor inconveniences that affect a few interventionists or recipients to severe issues that prevent all interventionists or recipients in a setting from completing part or all of the intervention. We propose 12 corresponding heuristics to guide EBPIs and implementation strategy designers in preventing and addressing these usability issues. CONCLUSIONS: Usability issues were prevalent in the studied EBPIs and implementation strategies. We recommend using the lens of usability evaluation to understand and address barriers to the effective use and reach of EBPIs and implementation strategies. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.2196/14990.

Modernizing Training in Psychotherapy Competencies With Adaptive Learning Systems: Proof of Concept.

PURPOSE: This proof-of-concept study assessed the feasibility, acceptability, and effectiveness of an intelligent tutoring system (ITS) as a classroom adjunct to improve training bachelor of social work (BSW) students in client engagement strategies. METHODS: We codeveloped the ITS with 11 undergraduate students and pilot tested it with six BSW students enrolled in a class on telephone-based cognitive behavioral therapy (tCBT). Student competencies in tCBT were assessed by expert review of role-plays. We also examined time spent using ITS and relation with competency. RESULTS: The majority of students (81.8%) in Wave 1 and all of the students who submitted role-plays in Wave 2 passed the clinical skills role-play. Students advancing through the ITS more quickly had better tCBT competency ratings than those progressing more slowly. DISCUSSION: One of the most challenging aspects of training is how to competently deliver evidence-based practices. ITS has the potential to streamline and scale such training.

Mental Health and the Perceived Usability of Digital Mental Health Tools Among Essential Workers and People Unemployed Due to COVID-19: Cross-sectional Survey Study.

BACKGROUND: COVID-19 has created serious mental health consequences for essential workers or people who have become unemployed as a result of the pandemic. Digital mental health tools have the potential to address this problem in a timely and efficient manner. OBJECTIVE: The purpose of this study was to document the extent of digital mental health tool (DMHT) use by essential workers and those unemployed due to COVID-19, including asking participants to rate the usability and user burden of the DMHT they used most to cope. We also explored which aspects and features of DMHTs were seen as necessary for managing stress during a pandemic by having participants design their own ideal DMHT. METHODS: A total of 2000 people were recruited from an online research community (Prolific) to complete a one-time survey about mental health symptoms, DMHT use, and preferred digital mental health features. RESULTS: The final sample included 1987 US residents that identified as either an essential worker or someone who was unemployed due to COVID-19. Almost three-quarters of the sample (1479/1987, 74.8%) reported clinically significant emotional distress. Only 14.2% (277/1957) of the sample used a DMHT to cope with stress associated with COVID-19. Of those who used DMHTs to cope with COVID-19, meditation apps were the most common (119/261, 45.6%). Usability was broadly in the acceptable range, although participants unemployed due to COVID-19 were less likely to report user burden with DMHTs than essential workers (t198.1=-3.89, P<.001). Individuals with emotional distress reported higher financial burden for their DMHT than nondistressed individuals (t69.0=-3.21, P=.01). When the sample was provided the option to build their own DMHT, the most desired features were a combination of mindfulness/meditation (1271/1987, 64.0%), information or education (1254/1987, 63.1%), distraction tools (1170/1987, 58.9%), symptom tracking for mood and sleep (1160/1987, 58.4%), link to mental health resources (1140/1987, 57.4%), and positive psychology (1131/1986, 56.9%). Subgroups by employment, distress, and previous DMHT use status had varied preferences. Of those who did not use a DMHT to cope with COVID-19, most indicated that they did not consider looking for such a tool to help with coping (1179/1710, 68.9%). CONCLUSIONS: Despite the potential need for DMHTs, this study found that the use of such tools remains similar to prepandemic levels. This study also found that regardless of the level of distress or even past experience using an app to cope with COVID-19, it is possible to develop a COVID-19 coping app that would appeal to a majority of essential workers and unemployed persons.

Using Real-world Data for Decision Support: Recommendations from a Primary Care Provider Survey.

INTRODUCTION: The use of data from wearable sensors, smartphones, and apps holds promise as a clinical decision-making tool in health and mental health in primary care medicine. The aim of this study was to determine provider perspectives about the utility of these data for building digitally based decision-making tools. METHODS: This mixed quantitative and qualitative cross-sectional survey of a convenience sample of primary-care clinicians at Kaiser Permanente Northwest was conducted between April and July 2019 online via Institute for Translational Health Sciences' Research Electronic Data Capture. Study outcomes were 1) attitudes toward digital data, 2) willingness to use digital data to support clinical decision making, and 3) concerns and recommendations about implementing a digital tool for clinical decision making. RESULTS: This sample of 131 clinicians was largely white (n = 98) female (n = 91) physicians (n = 86). Although respondents (75.7%, n = 87) had a positive attitude toward using digital tools in their practice, 88 respondents (67.3%) voiced concerns about the possible lack of clinical utility, suspected difficulty in integration with clinical workflows, and worried about the potential burden placed on patients. Participants indicated that the accuracy of the data in detecting the need for treatment adjustments would need to be high and the tool should be clinically tested. CONCLUSIONS: Primary care providers find value in collecting real-world patient data to assist in clinical decision making, provided such information does not interfere with provider workflow or impose undue burden on patients. In addition, digital tools will need to demonstrate high accuracy, be able to integrate into current clinical workflows, and maintain the privacy and security of patients' data.