Evaluating the effectiveness of concurrent sessions and counselors' attention allocation in online counseling.

OBJECTIVE: With its anonymity and accessibility, text-based online counseling has shown great potential in reaching people with mental health needs. One strategy adopted to meet the service gap is concurrent counseling, that is, each counselor attending to more than one client at a time. To date, there is no reported evidence supporting its rationality and effectiveness. This study investigated the potential opportunities, effectiveness, and caveats in concurrent service delivery and identified the optimal cutoff number of concurrent sessions while maintaining the quality of service at or above a set threshold. METHOD: We analyzed the transcript of 54,716 online counseling sessions from Open Up, a free, 24/7 text-based counseling service, to develop an attention score that measures the attention allocation of counselors and examined the impact of the counselor's attention allocation on client satisfaction and service outcomes. RESULTS: On average, compared to nonconcurrent sessions, concurrent sessions were longer, more likely to end prematurely, and had lower client satisfaction. We also identified an optimal attention score of approximately 0.4 (out of 1.0, which denotes full attention), which translates to two to three concurrent sessions. CONCLUSIONS: This study provides empirical evidence for the feasibility of conducting multiple text-based sessions concurrently without compromising service quality and client experience. Our method of measuring the counselor attention allocation offers a way to systematically assess and evaluate concurrent sessions. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Developing and validating a parser-based suicidality detection model in text-based mental health services.

BACKGROUND: Advances in text-mining can potentially aid online text-based mental health services in detecting suicidality. However, false positive remains a challenge. METHODS: Data of a free 24/7 online text-based counseling service in Hong Kong were used to develop a novel parser-based algorithm (PBSD) to detect suicidal ideation while minimizing false alarms. Sessions containing keywords related to suicidality were extracted (N = 1267). PBSD first applies a sentence parser to work out the grammatical structure of each sentence, including subject, object, dependent and modifier. Then a set of syntax rules were applied to judge if a flagged sentence is a true or false positive. Half of the sessions were randomly selected to train PBSD, the remaining were used as the test set. A standard keywords matching model was adopted as the baseline comparison. Accuracy and recall were reported to demonstrate models' performance. RESULTS: Of the 1267 sessions, 585 (46.2 %) were false alarms. The false alarms were categorized into four types: negation-induced false alarms (NIFA; 14 %), subject-induced false alarms (SIFA; 19 %), tense-induced false alarms (TIFA; 30 %), and other types of false alarms (OTFA; 37 %). PBSD significantly outperforms the baseline keywords matching model on accuracy (0.68 vs 0.53, 28.3 %). It successfully amended 36.8 % (105/297) lexicon matching-caused false alarms. The reduction on recall was marginal (1 vs 0.96, 4 %). CONCLUSIONS: The proposed model significantly improves the use of lexicon-based method by reducing false alarms and improving the accuracy of suicidality detection. It can potentially reduce unnecessary panic and distraction caused by false alarms among frontline service-providers.

Network-based prediction of the disclosure of ideation about self-harm and suicide in online counseling sessions.

BACKGROUND: In psychological services, the transition to the disclosure of ideation about self-harm and suicide (ISS) is a critical point warranting attention. This study developed and tested a succinct descriptor to predict such transitions in an online synchronous text-based counseling service. METHOD: We analyzed two years' worth of counseling sessions (N = 49,770) from Open Up, a 24/7 service in Hong Kong. Sessions from Year 1 (N = 20,618) were used to construct a word affinity network (WAN), which depicts the semantic relationships between words. Sessions from Year 2 (N = 29,152), including 1168 with explicit ISS, were used to train and test the downstream ISS prediction model. We divided and classified these sessions into ISS blocks (ISSBs), blocks prior to ISSBs (PISSBs), and non-ISS blocks (NISSBs). To detect PISSB, we adopted complex network approaches to examine the distance among different types of blocks in WAN. RESULTS: Our analyses find that words within a block tend to form a module in WAN and that network-based distance between modules is a reliable indicator of PISSB. The proposed model yields a c-statistic of 0.79 in identifying PISSB. CONCLUSIONS: This simple yet robust network-based model could accurately predict the transition point of suicidal ideation prior to its explicit disclosure. It can potentially improve the preparedness and efficiency of help-providers in text-based counseling services for mitigating self-harm and suicide.

In online counseling, the help-provider can often be engaging with several service users simultaneously. Therefore, new tools that could help to alert and assist the help-provider and increase their preparedness for getting further help for service users could be useful. In this study, we developed and tested a new tool that is designed to alert help-providers to the disclosure of self-harm and suicidal thoughts, based on the words that the service user has been typing. The tool is developed on the basis that word usage may have a specific pattern when suicidal thoughts are more likely to occur. We tested our tool using two years' worth of online counseling conversations and we show that our approach can help to predict the confession of suicidal thoughts. As such, we are taking a step forward in helping to improve these counseling services.

The Opportunities and Challenges of the First Three Years of Open Up, an Online Text-Based Counselling Service for Youth and Young Adults.

We present the opportunities and challenges of Open Up, a free, 24/7 online text-based counselling service to support youth in Hong Kong. The number of youths served more than doubled within the first three years since its inception in 2018 in response to increasing youth suicidality and mental health needs. Good practice models are being developed in order to sustain and further scale up the service. We discuss the structure of the operation, usage pattern and its effectiveness, the use of AI to improve users experience, and the role of volunteer in the operation. We also present the challenges in further enhancing the operation, calling for more research, especially on the identification of the optimal number of users that can be concurrently served by a counsellor, the effective approach to respond to a small percentage of repeated users who has taken up a disproportional volume of service, and the way to optimize the use of big data analytics and AI technology to enhance the service. These advancements will benefit not only Open Up but also similar services across the globe.

Detecting premature departure in online text-based counseling using logic-based pattern matching.

BACKGROUND: More so than face-to-face counseling, users of online text-based services might drop out from a session before establishing a clear closure or expressing the intention to leave. Such premature departure may be indicative of heightened risk or dissatisfaction with the service or counselor. However, there is no systematic way to identify this understudied phenomenon. PURPOSE: This study has two objectives. First, we developed a set of rules and used logic-based pattern matching techniques to systematically identify premature departures in an online text-based counseling service. Second, we validated the importance of premature departure by examining its association with user satisfaction. We hypothesized that the users who rated the session as less helpful were more likely to have departed prematurely. METHOD: We developed and tested a classification model using a sample of 575 human-annotated sessions from an online text-based counseling platform. We used 80% of the dataset to train and develop the model and 20% of the dataset to evaluate the model performance. We further applied the model to the full dataset (34,821 sessions). We compared user satisfaction between premature departure and completed sessions based on data from a post-session survey. RESULTS: The resulting model achieved 97% and 92% F1 score in detecting premature departure cases in the training and test sets, respectively, suggesting it is highly consistent with the judgment of human coders. When applied to the full dataset, the model classified 15,150 (43.5%) sessions as premature departure and the remaining 19,671 (56.5%) as completed sessions. Completed cases (15.2%) were more likely to fill the post-chat survey than premature departure cases (4.0%). Premature departure was significantly associated with lower perceived helpfulness and effectiveness in distress reduction. CONCLUSIONS: The model is the first that systematically and accurately identifies premature departure in online text-based counseling. It can be readily modified and transferred to other contexts for the purpose of risk mitigation and service evaluation and improvement.

A Local Review on the Use of a Bio-Psycho-Social Model in School-Based Mental Health Promotion.

Background: Schools are a key setting for student well-being promotion. Various school-based mental health programs have been implemented worldwide, with greater emphasis placed on psychological and social aspects. The bio-psycho-social model provides a holistic and integrated view of mental health based on theory and research evidence. Given the importance of considering all three dimensions in mental health promotion, this study explored reasons for the relative neglect of this approach by studying the early phase of school well-being program development and implementation. Method: In total, 77 Hong Kong government-funded student well-being programs implemented in 2000-2009 were reviewed for the use of biological, psychological, and social interventions. Questionnaires and interviews were conducted to explore program leaders' usage and views regarding theoretical frameworks and evidence-based practice and program evaluation. Challenges encountered in the initial stage of school well-being program development and implementation were identified and analyzed. Results: Of the 77 programs reviewed, only 5 addressed all three bio-psycho-social factors of mental health. A significantly greater number of programs addressed psychological (n = 63) and social (n = 40) factors compared to those that covered biological factors of mental health (n = 13). Of 24 program implementers who responded to the online survey, 75% claimed to have studied or applied a theoretical framework yet only 41.7% considered evidence-based practices to be important. The majority of interviewed participants valued their own practical experience over theory and research evidence. Many programs lacked rigorous evaluation of clear objectives and measurable outcomes, thus the mechanisms of change and program effectiveness were uncertain. Perceived barriers to program adoption and continuation were identified. Conclusion: This study highlighted a neglect of the biological contribution to mental health in school well-being promotion initiatives, possibly due to lack of theoretical knowledge and evidence-based practice among program leaders and implementers in the early phase of school mental health promotion. The bio-psycho-social model should therefore be recommended for student well-being programs as a holistic and integrated theory of mental health underpinning program objectives, mechanisms of change, and measurable outcomes. To develop effective practices in student well-being promotion, more thorough documentation, a rigorous evaluation framework, and support for frontline educators to evaluate their practices were recommended.