The Reach, Effectiveness, Adoption, Implementation, and Maintenance of Digital Mental Health Interventions for College Students: A Systematic Review.

PURPOSE OF REVIEW: We evaluated the impact of digital mental health interventions (DMHIs) for college students. We organized findings using the RE-AIM framework to include reach, effectiveness, adoption, implementation, and maintenance. RECENT FINDINGS: We conducted a systematic literature review of recent findings from 2019-2024. Our search identified 2,701 articles, of which 95 met inclusion criteria. In the reach domain, student samples were overwhelmingly female and White. In the effectiveness domain, over 80% of DMHIs were effective or partially effective at reducing their primary outcome. In the adoption domain, studies reported modest uptake for DMHIs. In the implementation and maintenance domains, studies reported high adherence rates to DMHI content. While recruitment methods were commonly reported, adaptations and costs of implementation and maintenance were rarely reported. DMHIs for college students are effective for many psychological outcomes. Future work should address diversifying samples and considering implementation in a variety of college settings.

A randomized controlled trial comparing brief online self-guided interventions for loneliness.

OBJECTIVE: Loneliness is a global health issue, but current loneliness interventions are not scalable enough to reach many who might benefit from them. Brief online interventions could greatly expand access to evidence-based loneliness interventions. METHOD: We conducted a preregistered three-armed trial (N = 908, ages 16-78) to compare three self-guided online interventions: a single-session intervention (SSI) for loneliness, a 3-week, three-session intervention for loneliness, and an active control supportive therapy SSI (https://ClinicalTrials.gov, ID: NCT05687162). RESULTS: Loneliness decreased between baseline and Week 8 across all conditions (b = -5.80; d = -0.55; 95% CI [-0.62, -0.47]; p < .01), but did not decrease significantly more in those assigned to either the loneliness SSI (b = -1.27; d = -0.12; 95% CI [-0.30, 0.06]; p = .20) or the 3-week intervention (b = -0.93; d = -0.09; 95% CI [-0.27, 0.09]; p = .34) than those assigned to the control SSI. Participants found all three interventions acceptable but rated both loneliness interventions as more acceptable than the control (p < .01). Far more participants completed the 10-min control SSI (86.6%) and 20-min loneliness SSI (69.4%) than the 60-min 3-week intervention (14.9%). CONCLUSIONS: An SSI for loneliness was not significantly less effective than a longer loneliness intervention and had a much higher completion rate. Yet, against our hypotheses, neither loneliness intervention reduced loneliness more than an active control SSI did. Future work should aim to design more effective SSIs for loneliness and identify populations for which SSIs might be most helpful. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Leveraging Feedback From Families of Children With Autism to Create Digital Support for Service Navigation: Descriptive Study.

BACKGROUND: It is difficult for families to navigate and access services for their children with autism. Barriers to service access are compounded among families from low-resourced backgrounds. OBJECTIVE: The purpose of our study was to explore the development of an app to facilitate access to services among families of children with autism from low-resourced backgrounds. Our specific aims were to explore feedback from an advisory board about the app and to explore feedback from navigators about the app. METHODS: Via a multistage codevelopment process, we elicited feedback from 5 key parties: the research team, a community organization, the app development team, the advisory board, and family navigators. Collectively, 36 individuals provided feedback about the development of the app via individual interviews, focus groups, observations, and surveys. The key features of the app included a dashboard showing the service needs of the family and related resources, a messaging feature between the family, the navigator, and the supervisor, and a fidelity checklist and evaluation feature. RESULTS: The advisory board provided feedback about the app to increase its user-friendliness, include the ability to develop an action plan, improve the identification of needed services, and add information about service providers. Navigators suggested that the app should connect navigators to one another, have a clearer purpose for the notes section, and reflect an easier log-in process. Navigators also wanted training to role-play using the app. After participating in a role play using the app, navigators reported significantly more satisfaction with the app and greater usefulness (P<.001). CONCLUSIONS: Our work sheds light on the importance of eliciting feedback from end users, especially users who are often overlooked by the research community and app developers. Further, it is important to elicit feedback in multiple ways to improve the app.

Individual and organizational outcomes of engaging peers in the cocreation of digital mental health interventions.

Within mental health services, persons in recovery from their own experiences of mental health challenges (peers) are increasingly being trained to provide peer support. This study describes individual and organizational outcomes related to engaging peers in a multisite demonstration project in California that sought to integrate them as cocreators throughout planning and implementation of digital mental health interventions. We collected data from key informants across 11 sites. Quarterly online surveys invited key informants to report perceived outcomes of the peer component. Biannual interviews elicited details regarding survey-reported outcomes. Quantitative data provided indications of outcome prevalence and consistency, and quotes from the interviews illustrated the complex realities underlying survey responses. One hundred three quarterly surveys and 39 biannual interviews were completed between Summer 2020 and Fall 2022. Key informants reported diverse outcomes, including integration of peer input into local decision making, mental health benefits to peers and community members, reduced workplace mental health stigma, and new cross-site collaborations. Five sites reported outcomes with greater consistency compared to the other six sites. Reports of increased peer visibility in the workplace coincided with reports of reduced stigma and increased value of peer input by mental health professionals. This study offers encouragement for the potential positive impact of engaging peers as cocreators of mental health interventions. Data suggest integrating peers does not increase mental health stigma and may instead result in various positive outcomes. The degree to which these outcomes manifest in a specific setting, however, may vary. Future research should seek to identify contextual factors that support actualization of positive outcomes. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Facilitators of and barriers to County Behavioral Health System Transformation and Innovation: an interview study.

BACKGROUND: Inadequate and inequitable access to quality behavioral health services and high costs within the mental health systems are long-standing problems. System-level (e.g., fee-for-service payment model, lack of a universal payor) and individual factors (e.g., lack of knowledge of existing resources) contribute to difficulties in accessing resources and services. Patients are underserved in County behavioral health systems in the United States. Orange County's (California) Behavioral Health System Transformation project sought to improve access by addressing two parts of their system: developing a template for value-based contracts that promote payor-agnostic care (Part 1); developing a digital platform to support resource navigation (Part 2). Our aim was to evaluate facilitators of and barriers to each of these system changes. METHODS: We collected interview data from County or health care agency leaders, contracted partners, and community stakeholders. Themes were informed by the Consolidated Framework for Implementation Research. RESULTS: Five themes were identified related to behavioral health system transformation, including 1) aligning goals and values, 2) addressing fit, 3) fostering engagement and partnership, 4) being aware of implementation contexts, and 5) promoting communication. A lack of fit into incentive structures and changing state guidelines and priorities were barriers to contract development. Involving diverse communities to inform design and content facilitated the process of developing digital tools. CONCLUSIONS: The study highlights the multifaceted factors that help facilitate or hinder behavioral health system transformation, such as the need for addressing systematic and process behaviors, leveraging the knowledge of leadership and community stakeholders, fostering collaboration, and adapting to implementation contexts.

Variability in the Integration of Peers in a Multi-site Digital Mental Health Innovation Project.

Peer support specialists ("peers") who have the lived experience of, and are in recovery from, mental health challenges are increasingly being integrated into mental health care as a reimbursable service across the US. This study describes the ways peers were integrated into Help@Hand, a multi-site innovation project that engaged peers throughout efforts to develop and offer digital mental health interventions across counties/cities ("sites") in California. Using a mixed methods design, we collected quantitative data via quarterly online surveys, and qualitative data via semi-annual semi-structured phone interviews with key informants from Help@Hand sites. Quantitative data were summarized as descriptive findings and qualitative data from interviews were analyzed using rapid qualitative analysis methods. In the final analytic phase, interview quotes were used to illustrate the complex realities underlying quantitative responses. 117 quarterly surveys and 46 semi-annual interviews were completed by key informants from 14 sites between September 2020 and January 2023. Peers were integrated across diverse activities for support and implementation of digital mental health interventions, including development of training and educational materials (78.6% of sites), community outreach (64.3%), technology testing (85.7%), technology piloting (90.9%), digital literacy training (71.4%), device distribution (63.6%), technical assistance (72.7%), and cross-site collaboration (66.7%). Peer-engaged activities shifted over time, reflecting project phases. Peer-provided digital literacy training and technology-related support were key ingredients for project implementations. This study indicates the wide range of ways peers can be integrated into digital mental health intervention implementations. Considering contextual readiness for peer integration may enhance their engagement into programmatic activities.

Mental Health Apps for Children and Adolescents: A Clinician-Friendly Review.

Mobile apps for mental health and wellness (MH apps) have the potential to support youth mental health, expanding access to the large proportion of youth with mental health concerns who do not access formal treatment. Survey data suggest that young people are highly interested in MH apps, with minoritized youth (eg, LGBTQ individuals) and youth with elevated depressive symptoms reporting especially high rates of downloading MH apps.1 In addition, systematic reviews and meta-analyses suggest that MH apps can be effective.2 Although many popular MH apps do not have direct empirical evidence supporting their efficacy,3 they often include elements of empirically supported treatments for children and adolescents.4 The MH app space moves at a fast pace, making it difficult to stay up-to-date. More than 10,000 MH apps exist, with hundreds released each year.5 Even interested clinicians may, understandably, not have time to sort through the large number of apps, scientific papers, and app directories. Furthermore, these resources are rarely designed to offer clear, actionable advice for clinicians. Clinicians could benefit from information and guidance to help patients safely navigate MH apps and to best use MH apps in treatment. Here, we provide 3 key points about youth MH apps ("takeaways"), each with an actionable implication for clinicians ("action items"). We also provide examples of specific questions for clinicians to support their use of MH apps with children and adolescents (Table 1). More information regarding where we derived these recommendations is available in Supplement 1, available online.

Effectiveness-implementation hybrid trial of Spanish language, digital cognitive-behavioral therapy (dCBT) intervention for depression and anxiety - protocol for the SUPERA (SUpport from PEeRs to expand Access) study.

BACKGROUND: Limited English Proficiency (LEP) Latinxs experience a longer duration of untreated depression and anxiety. LEP Latinxs have difficulty accessing mental healthcare due to insufficient Spanish-speaking behavioral/mental health clinicians to meet demand. These under-resourced healthcare systems are less likely to be the site for the implementation of innovations. Digital interventions can provide an effective option for overcoming these barriers; yet, when digital evidence-based treatments are available, uptake and engagement is often low. This manuscript presents the protocol for the SUPERA (SUpport from PEeRs to expand Access) study which will evaluate the implementation of an evidence-based, Spanish language, digital cognitive-behavioral therapy (dCBT) intervention (i.e., SilverCloud) in safety-net primary care clinics for LEP Latinx patients with depression or anxiety. METHODS: We will conduct an effectiveness-implementation hybrid trial (Type 2) design comparing engagement and clinical outcomes in two modalities of dCBT delivery (peer-supported vs. unsupported). We will also compare provider-level outreach (using a clinic patient registry) versus inreach (traditional provider referral) to compare rates of initiation, completion, and cost. Participants will be 426 LEP Latinx adults ≥18 years of age, PHQ-9 ≥ 10 or GAD-7 ≥ 8, with access to the internet via smartphone, and not currently receiving individual psychotherapy. We will collect baseline, post-intervention (8 weeks), and follow up (3 months) data. CONCLUSION: The long-term goal of this research is to aid in the implementation of digital mental health interventions that can be sustainably implemented in low-resourced settings, while reducing the reliance on professionals, overcoming workforce deficits, and increasing relevance for diverse populations.

Real-World Adoption of Mental Health Support Among Adolescents: Cross-Sectional Analysis of the California Health Interview Survey.

OBJECTIVE: We aim to examine: (a) the extent to which patterns of adoption of counseling services and digital mental health interventions (DMHIs) shifted in recent years (2019-2021); (b) the impact of distress on adoption of mental health support; and (c) reasons related to adolescents' low adoption of DMHIs when experiencing distress. METHODS: Data were from three cohorts of adolescents aged 12-17 years (n = 847 in 2019; n = 1,365 in 2020; n = 1,169 in 2021) recruited as part of the California Health Interview Survey. We estimated logistic regression models to examine the likelihood of using mental health support as a function of psychological distress, sociodemographic characteristics, and cohorts. We also analyzed adolescents' self-reported reasons for not trying DMHIs as a function of distress. RESULTS: The proportion of adolescents reporting elevated psychological distress (∼50%) was higher than those adopting counseling services (<20%) or DMHIs (<10%). A higher level of distress was associated with a greater likelihood of receiving counseling (OR = 1.15), and using DMHIs to connect with a professional (Odds ratio (OR) = 1.11) and for self-help (OR = 1.17). Among those experiencing high distress, adolescents' top reason for not adopting an online tool was a lack of perceived need (19.2%). CONCLUSION: Adolescents' main barriers to DMHI adoption included a lack of perceived need, which may be explained by a lack of mental health literacy. Thoughtful marketing and dissemination efforts are needed to increase mental health awareness and normalize adoption of counseling services and DMHIs.

Cultural Responsivity in Technology-Enabled Services: Integrating Culture Into Technology and Service Components.

Technology-enabled services (TESs) are clinical interventions that combine technological and human components to provide health services. TESs for mental health are efficacious in the treatment of anxiety and depression and are currently being offered as frontline treatments around the world. It is hoped that these interventions will be able to reach diverse populations across a range of identities and ultimately decrease disparities in mental health treatment. However, this hope is largely unrealized. TESs include both technology and human service components, and we argue that cultural responsivity must be considered in each of these components to help address existing treatment disparities. To date, there is limited guidance on how to consider cultural responsivity within these interventions, including specific targets for the development, tailoring, or design of the technologies and services within TESs. In response, we propose a framework that provides specific recommendations for targets based on existing models, both at the technological component level (informed by the Behavioral Intervention Technology Model) and the human support level (informed by the Efficiency Model of Support). We hope that integrating culturally responsive considerations into these existing models will facilitate increased attention to cultural responsivity within TESs to ensure they are ethical and responsive for everyone.

Exploring User Perspectives of and Ethical Experiences With Teletherapy Apps: Qualitative Analysis of User Reviews.

BACKGROUND: Teletherapy apps have emerged as a promising alternative to traditional in-person therapy, especially after the COVID-19 pandemic, as they help overcome a range of geographical and emotional barriers to accessing care. However, the rapid proliferation of teletherapy apps has occurred in an environment in which development has outpaced the various regulatory and ethical considerations of this space. Thus, researchers have raised concerns about the ethical implications and potential risks of teletherapy apps given the lack of regulation and oversight. Teletherapy apps have distinct aims to more directly replicate practices of traditional care, as opposed to mental health apps, which primarily provide supplemental support, suggesting a need to examine the ethical considerations of teletherapy apps from the lens of existing ethical guidelines for providing therapy. OBJECTIVE: In this study, we examined user reviews of commercial teletherapy apps to understand user perceptions of whether and how ethical principles are followed and incorporated. METHODS: We identified 8 mobile apps that (1) provided teletherapy on 2 dominant mobile app stores (Google Play and Apple App Store) and (2) had received >5000 app reviews on both app stores. We wrote Python scripts (Python Software Foundation) to scrape user reviews from the 8 apps, collecting 3268 user reviews combined across 2 app stores. We used thematic analysis to qualitatively analyze user reviews, developing a codebook drawing from the ethical codes of conduct for psychologists, psychiatrists, and social workers. RESULTS: The qualitative analysis of user reviews revealed the ethical concerns and opportunities of teletherapy app users. Users frequently perceived unprofessionalism in their teletherapists, mentioning that their therapists did not listen to them, were distracted during therapy sessions, and did not keep their appointments. Users also noted technical glitches and therapist unavailability on teletherapy apps that might affect their ability to provide continuity of care. Users held varied opinions on the affordability of those apps, with some perceiving them as affordable and others not. Users further brought up that the subscription model resulted in unfair pricing and expressed concerns about the lack of cost transparency. Users perceived that these apps could help promote access to care by overcoming geographical and social constraints. CONCLUSIONS: Our study suggests that users perceive commercial teletherapy apps as adhering to many ethical principles pertaining to therapy but falling short in key areas regarding professionalism, continuity of care, cost fairness, and cost transparency. Our findings suggest that, to provide high-quality care, teletherapy apps should prioritize fair compensation for therapists, develop more flexible and transparent payment models, and invest in measures to ensure app stability and therapist availability. Future work is needed to develop standards for teletherapy and improve the quality and accessibility of those services.

Clinical science and practice in the age of large language models and generative artificial intelligence.

In this article, Schueller and Morris discuss the recent advances made from large language models (LLMs) and generative artificial intelligence (AI). These advances include supporting humans to provide better interventions, understanding processes in clinical interventions, and providing ethical considerations for the use of generative AI in clinical research and practice. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

A two-phase, prescriptive comparative effectiveness study to optimize the treatment of co-occurring insomnia and depression with digital interventions.

BACKGROUND: Insomnia and depression frequently co-occur. Significant barriers preclude a majority of patients from receiving first line treatments for both disorders in a sequential treatment episode. Although digital versions of cognitive behavioral therapy for insomnia (CBTI) and for depression (CBTD) hold some promise to meet demand, especially when paired with human support, it is unknown whether heterogeneity of treatment effects exist, such that some patients would be optimally treated with single or sequential interventions. OBJECTIVE: Describe the protocol for a two-phase, prescriptive comparative effectiveness study to develop and evaluate an individualized intervention rule (IIR) for prescribing the optimal digital treament of co-occurring insomnia and depression. METHODS: The proposed sample size is 2300 U.S. military veterans with insomnia and depression recruited nationally (Phase 1 = 1500; Phase 2 = 800). In each phase, the primary endpoint will be remission of both depression and insomnia 3 months following a 12-week intervention period. Phase 1 is a 5-arm randomized trial: two single digital interventions (CBT-I or CBT-D); two sequenced interventions (CBT-I + D or CBT-D + I); and a mood monitoring control condition. A cutting-edge ensemble machine learning method will be used to develop the IIR. Phase 2 will evaluate the IIR by randomizing participants with equal allocation to either the IIR predicted optimal intervention for that individual or by randomization to one the four CBT interventions. RESULTS: Study procedures are ongoing. Results will be reported in future manuscripts. CONCLUSION: The study will generate evidence on the optimal scalable approach to treat co-occurring insomnia and depression.

Understanding Users' Experiences of a Novel Web-Based Cognitive Behavioral Therapy Platform for Depression and Anxiety: Qualitative Interviews From Pilot Trial Participants.

BACKGROUND: Digital mental health interventions (DMHIs) can help bridge the gap between the demand for mental health care and availability of treatment resources. The affordances of DMHIs have been proposed to overcome barriers to care such as accessibility, cost, and stigma. Despite these proposals, most evaluations of the DMHI focus on clinical effectiveness, with less consideration of users' perspectives and experiences. OBJECTIVE: We conducted a pilot randomized controlled trial of "Overcoming Thoughts," a web-based platform that uses cognitive and behavioral principles to address depression and anxiety. The "Overcoming Thoughts" platform included 2 brief interventions-cognitive restructuring and behavioral experimentation. Users accessed either a version that included asynchronous interactions with other users ("crowdsourced" platform) or a completely self-guided version (control condition). We aimed to understand the users' perspectives and experiences by conducting a subset of interviews during the follow-up period of the trial. METHODS: We used purposive sampling to select a subset of trial participants based on group assignment (treatment and control) and symptom improvement (those who improved and those who did not on primary outcomes). We conducted semistructured interviews with 23 participants during the follow-up period that addressed acceptability, usability, and impact. We conducted a thematic analysis of the interviews until saturation was reached. RESULTS: A total of 8 major themes were identified: possible opportunities to expand the platform; improvements in mental health because of using the platform; increased self-reflection skills; platform being more helpful for certain situations or domains; implementation of skills into users' lives, even without direct platform use; increased coping skills because of using the platform; repetitiveness of platform exercises; and use pattern. Although no differences in themes were found among groups based on improvement status (all P values >.05, ranging from .12 to .86), there were 4 themes that differed based on conditions (P values from .01 to .046): helpfulness of self-reflection supported by an exercise summary (greater in control); aiding in slowing thoughts and feeling calmer (greater in control); overcoming patterns of avoidance (greater in control); and repetitiveness of content (greater in the intervention). CONCLUSIONS: We identified the different benefits that users perceived from a novel DMHI and opportunities to improve the platform. Interestingly, we did not note any differences in themes between those who improved and those who did not, but we did find some differences between those who received the control and intervention versions of the platform. Future research should continue to investigate users' experiences with DMHIs to better understand the complex dynamics of their use and outcomes.

Facilitators of and Barriers to Integrating Digital Mental Health Into County Mental Health Services: Qualitative Interview Analyses.

BACKGROUND: Digital mental health interventions (DMHIs) represent a promising solution to address the growing unmet mental health needs and increase access to care. Integrating DMHIs into clinical and community settings is challenging and complex. Frameworks that explore a wide range of factors, such as the Exploration, Preparation, Implementation, Sustainment (EPIS) framework, can be useful for examining multilevel factors related to DMHI implementation efforts. OBJECTIVE: This paper aimed to identify the barriers to, facilitators of, and best practice recommendations for implementing DMHIs across similar organizational settings, according to the EPIS domains of inner context, outer context, innovation factors, and bridging factors. METHODS: This study stems from a large state-funded project in which 6 county behavioral health departments in California explored the use of DMHIs as part of county mental health services. Our team conducted interviews with clinical staff, peer support specialists, county leaders, project leaders, and clinic leaders using a semistructured interview guide. The development of the semistructured interview guide was informed by expert input regarding relevant inner context, outer context, innovation factors, and bridging factors in the exploration, preparation, and implementation phases of the EPIS framework. We followed a recursive 6-step process to conduct qualitative analyses using inductive and deductive components guided by the EPIS framework. RESULTS: On the basis of 69 interviews, we identified 3 main themes that aligned with the EPIS framework: readiness of individuals, readiness of innovations, and readiness of organizations and systems. Individual-level readiness referred to the extent to which clients had the necessary technological tools (eg, smartphones) and knowledge (digital literacy) to support the DMHI. Innovation-level readiness pertained to the accessibility, usefulness, safety, and fit of the DMHI. Organization- and system-level readiness concerned the extent to which providers and leadership collectively held positive views about DMHIs as well as the extent to which infrastructure (eg, staffing and payment model) was appropriate. CONCLUSIONS: The successful implementation of DMHIs requires readiness at the individual, innovation, and organization and system levels. To improve individual-level readiness, we recommend equitable device distribution and digital literacy training. To improve innovation readiness, we recommend making DMHIs easier to use and introduce, clinically useful, and safe and adapting them to fit into the existing client needs and clinical workflow. To improve organization- and system-level readiness, we recommend supporting providers and local behavioral health departments with adequate technology and training and exploring potential system transformations (eg, integrated care model). Conceptualizing DMHIs as services allows the consideration of both the innovation characteristics of DMHIs (eg, efficacy, safety, and clinical usefulness) and the ecosystem around DMHIs, such as individual and organizational characteristics (inner context), purveyors and intermediaries (bridging factor), client characteristics (outer context), as well as the fit between the innovation and implementation settings (innovation factor).