An early economic evaluation of Kooth, a web-based mental health platform for children and young people with emerging mental health needs.

Background: "Kooth" is a web-based mental health platform commissioned by the National Health Service (NHS), local authorities, charities, and businesses in the UK. The platform gives children and young people (CYP) access to an online community of peers and a team of counsellors. This study reports an early economic evaluation of the potential benefits of Kooth in the UK. Methods: An early evidence cost calculator was built to estimate the potential costs and savings of implementing Kooth from a UK NHS and crime sector perspective. A decision tree structure was used to track the progress of CYP with emerging mental health needs (EMHN), comparing CYP with access to Kooth to CYP without access to Kooth. The model implemented a 12-month time horizon and followed a typical Kooth contract in relation to costing, engagement, and CYP demographics. Results: The base case results followed a cohort of 2160 CYP. The results of the cost calculator estimated that engagement with Kooth is associated with a cost saving of £469,237 to the NHS across a 12-month time horizon, or £236.15 per CYP with an EMHN. From a combined NHS and UK crime sector perspective, the cost savings increased to £489,897, or £246.54 per CYP with an EMHN. The largest cost savings were provided by an estimated reduction of 5346 GP appointments and 298 antidepressant prescriptions. For this cohort, the model predicted that engagement with Kooth averted 6 hospitalisations due to suicidal ideation and 13 hospitalisations due to self-harm. Furthermore, the number of smokers and binge drinkers was reduced by 20 and 24, respectively. When a crime sector perspective was taken, 3 crimes were averted. Discussion: This early model demonstrates that Kooth has the potential to be a cost-saving intervention from both an NHS and a combined NHS and UK crime sector perspective. Cost savings were provided through aversion in clinical and social outcomes. The model used a conservative approach to balance the uncertainty around assumptions of the intermediate outcomes (GP and medication use). However, it is limited by a paucity of costing data and published evidence relating to the impact of digital mental health platforms.

Patterns of engagement in a digital mental health service during COVID-19: a cohort study for children and young people.

Introduction: The COVID-19 pandemic increased public use of digital mental health technologies. However, little is known about changes in user engagement with these platforms during the pandemic. This study aims to assess engagement changes with a digital mental healthcare service during COVID-19. Methods: A cohort study based on routinely collected service usage data from a digital mental health support service in the United Kingdom. Returning users aged 14-25 years that interacted for a maximum of two months were included. The study population was divided into pre-COVID and COVID cohorts. Demographic and usage information between cohorts were compared and usage clusters were identified within each cohort. Differences were tested using Chi-squared, two-sample Kolmogorov-Smirnov tests and logit regressions. Results: Of the 624,103 users who joined the service between May 1, 2019, and October 1, 2021, 18,889 (32.81%) met the inclusion criteria: 5,048 in the pre-COVID cohort and 13,841 in the COVID cohort. The COVID cohort wrote more journals; maintained the same focus on messaging practitioners, posting discussions, commenting on posts, and having booked chats; and engaged less in writing journals, setting personal goals, posting articles, and having ad-hoc chats. Four usage profiles were identified in both cohorts: one relatively disengaged, one focused on contacting practitioners through chats/messages, and two broadly interested in writing discussions and comments within the digital community. Despite their broad similarities, usage patterns also exhibited differences between cohorts. For example, all four clusters had over 70% of users attempting to have ad-hoc chats with practitioners in the pre-COVID cohort, compared to one out of four clusters in the COVID cohort. Overall, engagement change patterns during the COVID-19 pandemic were not equal across clusters. Sensitivity analysis revealed varying strength of these defined clusters. Discussion: Our study identified changes in user activity and engagement behavior within a digital mental healthcare service during the COVID-19 pandemic. These findings suggest that usage patterns within digital mental health services may be susceptible to change in response to external events such as a pandemic. Continuous monitoring of engagement patterns is important for informed design and personalized interventions.

Examining concurrent validity and item selection of the Session Wants and Needs Outcome Measure (SWAN-OM) in a children and young people web-based therapy service.

Background: Single-session mental health interventions are frequently attended by children and young people (CYP) in both web-based and face-to-face therapy settings. The Session "Wants" and "Needs" Outcome Measure (SWAN-OM) is an instrument developed in a web-based therapy service to overcome the challenges of collecting outcomes and experiences of single-session therapies (SSTs). It provides pre-defined goals for the session, selected by the young person prior to the intervention, on which progress toward achievement is scored at the end of the session. Objective: The objective of this study was to evaluate the instrument's psychometric properties, including concurrent validity against three other frequently used outcome and experience measures, at a web-based and text-based mental health service. Methods: The SWAN-OM was administered for a period of 6 months to 1,401 CYP (aged 10-32 years; 79.3% white; 77.59% female) accessing SST on a web-based service. Item correlations with comparator measures and hierarchical logistic regressions to predict item selection were calculated for concurrent validity and psychometric exploration. Results: The most frequently selected items were "Feel better" (N = 431; 11.61%) and "Find ways I can help myself" (N = 411; 11.07%); unpopular items were "Feel safe in my relationships" (N = 53; 1.43%) and "Learn the steps to achieve something I want" (N = 58; 1.56%). The SWAN-OM was significantly correlated with the Experience of Service Questionnaire, particularly the item "Feel better" [rs(109) = 0.48, p < 0.001], the Youth Counseling Impact Scale, particularly the item "Learn the steps to achieve something I want" [rs(22) = 0.76, p < 0.001], and the Positive and Negative Affect Schedule, particularly the items "Learn how to feel better" [rs(22) = 0.72, p < 0.001] and "Explore how I feel" [rs(70) = -0.44, p < 0.001]. Conclusion: The SWAN-OM demonstrates good concurrent validity with common measures of outcome and experience. Analysis suggests that lesser-endorsed items may be removed in future iterations of the measure to improve functionality. Future research is required to explore SWAN-OM's potential to measure meaningful change in a range of therapeutic settings.

Alienated and unsafe: Experiences of the first national UK COVID-19 lockdown for vulnerable young people (aged 11-24 years) as revealed in Web-based therapeutic sessions with mental health professionals.

Background: The COVID-19 pandemic and subsequent lockdowns have disproportionately affected young people, and those who are vulnerable are disadvantaged further. Here, we seek to understand the experiences of vulnerable young people accessing Web-based therapeutic support during the pandemic and early lockdown, as revealed through the observations of mental health professionals. Methods: Four focus groups with 12 professionals from a digital mental health service were conducted to understand the experiences of vulnerable young people during the pandemic lockdown. Workshops with young people with diverse experiences resulted in the co-design of the focus group topic guide and the interpretation and validation of analysis. The experiential inductive-deductive framework of thematic analysis was used to analyse the workshop transcripts. Results: Four main themes and subsequent subthemes were identified: escalation of risk; the experience and consequence of loss; feeling supported and empowered; and feeling separate and isolated. Conclusions: Findings reflect early data that suggest that those with existing vulnerability face an increased risk of poor outcomes through the pandemic and the restrictions of lockdown, but evidence is also provided of positive outcomes from lockdown and its function as a catalyst for change. Results reinforce the need for focused support for vulnerable young people as we emerge from lockdown, and point to supportive and protective factors of relevance to online and offline support provision.

Engagement of individuals aged 14-25 years with a digital mental health service during the COVID-19 pandemic in the UK: a cohort study of service usage data.

BACKGROUND: Public use of digital mental health technologies has informed several studies focusing on patterns of engagement within user-led digital support systems. General engagement with these services has increased since the start of the COVID-19 pandemic. However, to the best of our knowledge, this is the first study to explore how user engagement patterns with these platforms changed during the pandemic. METHODS: In this cohort study, we included individuals aged 14-25 years who signed up to the online mental health platform Kooth in the UK before the pandemic (May 1, 2019 to March 10, 2020) and during the pandemic (March 11, 2020 to Oct 31, 2021). Routinely collected usage data were assessed for consenting users who returned to Kooth at least once after sign-up. We used logistic regression to compare service component usage and overall engagement across cohorts. FINDINGS: 5048 individuals (mean sign-ups per day 9·0 [12·0]) were included in the pre-pandemic cohort and 13 841 (mean sign-ups per day 11·5 [13·9]) in the pandemic cohort, representing a significant increase in sign-ups per day during the pandemic compared with the pre-pandemic period (Welch's t-test p=0·0001). Compared with users pre-pandemic, users during the pandemic were less likely to have a drop-in chat with a practitioner (odds ratio [OR] 0·50 [95% CI 0·47-0·54], p<0·0001) but were more likely to journal (1·92 [1·77-2·07], p<0·0001) and comment on existing content (1·10 [1·03-1·18], p=0·008). Users who signed up to Kooth during the pandemic and requested a chat with a practitioner (7816 [56·5%] of 13 841 users) were also less likely to receive a chat than users who requested a chat pre-pandemic (3791 [75·1%] of 5048 users; OR 0·68 [0·63-0·74], p<0·0001). INTERPRETATION: During the COVID-19 pandemic, people with the ability and resources to access digital mental health support were diverted from face-to-face services to digital mental health services. This change might have resulted in an unmet need for drop-in practitioner chats, with users probably resorting to asynchronous support through community interaction or journaling. FUNDING: Kooth Digital Health.

The design and development of an experience measure for a peer community moderated forum in a digital mental health service.

Online digital mental health communities can contribute to users' mental health positively and negatively. Yet the measurement of experience, outcomes and impact mechanisms relating to digital mental health communities is difficult to capture. In this paper we demonstrate the development of an online experience measure for a specific children and young people's community forum inside a digital mental health service. The development of the Peer Online Community Experience Measure (POCEM) is informed by a multi-phased design: (i) item reduction through Estimate-Talk-Estimate modified Delphi methods, (ii) user testing with think-aloud protocols and (iii) a pilot study within the digital service community to explore observational data within the platform. Experts in the field were consulted to help reduce the items in the pool and to check their theoretical coherence. User testing workshops helped to inform the usability appearance, wording, and purpose of the measure. Finally, the pilot results highlight completion rates, differences in scores for age and roles and "relate to others", as the most frequent domain mechanism of support for this community. Outcomes frequently selected show the importance of certain aspects of the community, such as safety, connection, and non-judgment previously highlighted in the literature. Experience measures like this one could be used as indicators of active therapeutic engagement within the forum community and its content but further research is required to ascertain its acceptability and validity. Multi-phased approaches involving stakeholders and user-centred design activities enhances the development of digitally enabled measurement tools.

The Session Wants and Need Outcome Measure: The Development of a Brief Outcome Measure for Single-Sessions of Web-Based Support.

Single-session, brief interventions in therapy for young people make up a large proportion of service provision, including in digital mental health settings. Current nomothetic mental health measures are not specifically designed to capture the benefit or 'change' directly related to these brief interventions. As a consequence, we set out to design an outcome measure to concretely demonstrate the value of single-session interventions. The Session Wants and Needs Outcome Measure (SWAN-OM) aims to capture in-session goals and focuses on being user-centric, elements critical to the success of single-session and brief interventions which typically are asset-based and solution-focused. We describe the 4-stage process that was followed to develop this measure: (I) classical item generation and development, (II) content and (III) face validity pilot testing, and (IV) a user-experience approach with young people using framework analysis. This final stage was critical to ensure the integration of this outcome tool into a web-based digital therapy setting, a context which adds another layer of design complexity to item and measure development. This iterative methodology was used to overcome the challenges encountered and to place the needs of the young people and service practitioners at the centre of the design process, thus ensuring measure usability. To end, we highlight the main lessons learnt from engaging in this design process. Specifically, the needs of a measure for single-session interventions are considered, before outlining the learning associated with integrating the measure into a digital mental health platform. Both of these areas are emerging fields and, as such, this study contributes to our understanding of how an idiographic patient outcome theory driven measure can be created for use in a web-based digital mental health therapy service.

Intact strategic retrieval processes in older adults: no evidence for age-related deficits in source-constrained retrieval.

Aging is thought to involve impairments to cognitive control functions that support episodic memory, for example by enabling people to strategically constrain their retrieval search towards a specific context ("source") in order to facilitate retrieval of goal-relevant memories. The "memory-for-foils" paradigm investigates source-constrained retrieval by assessing whether incidental encoding of new foils during an old/new recognition test differs depending on the type of processing that was previously used during study of the old items in the test. If it does, it suggests that people process foils differently as a result of engaging in source-constrained retrieval attempts. Young adults typically show differences in incidental encoding foils, but such differences have not been found in older adults. Here, we compared source-constrained retrieval and reward effects on incidental foil encoding between younger and older adults, to assess if age-related reductions in strategic retrieval processing are accompanied by differences in responsiveness to external rewards. The results showed only minor effects of rewards on memory processing, in younger adults only. Contrary to prior findings, older adults had equivalent overall memory performance and spontaneously constrained retrieval to the same extent as the young group, showing that aging-related impairments to strategic retrieval processes are not inevitable.