Automated Behavioral Coding to Enhance the Effectiveness of Motivational Interviewing in a Chat-Based Suicide Prevention Helpline: Secondary Analysis of a Clinical Trial.

BACKGROUND: With the rise of computer science and artificial intelligence, analyzing large data sets promises enormous potential in gaining insights for developing and improving evidence-based health interventions. One such intervention is the counseling strategy motivational interviewing (MI), which has been found effective in improving a wide range of health-related behaviors. Despite the simplicity of its principles, MI can be a challenging skill to learn and requires expertise to apply effectively. OBJECTIVE: This study aims to investigate the performance of artificial intelligence models in classifying MI behavior and explore the feasibility of using these models in online helplines for mental health as an automated support tool for counselors in clinical practice. METHODS: We used a coded data set of 253 MI counseling chat sessions from the 113 Suicide Prevention helpline. With 23,982 messages coded with the MI Sequential Code for Observing Process Exchanges codebook, we trained and evaluated 4 machine learning models and 1 deep learning model to classify client- and counselor MI behavior based on language use. RESULTS: The deep learning model BERTje outperformed all machine learning models, accurately predicting counselor behavior (accuracy=0.72, area under the curve [AUC]=0.95, Cohen κ=0.69). It differentiated MI congruent and incongruent counselor behavior (AUC=0.92, κ=0.65) and evocative and nonevocative language (AUC=0.92, κ=0.66). For client behavior, the model achieved an accuracy of 0.70 (AUC=0.89, κ=0.55). The model's interpretable predictions discerned client change talk and sustain talk, counselor affirmations, and reflection types, facilitating valuable counselor feedback. CONCLUSIONS: The results of this study demonstrate that artificial intelligence techniques can accurately classify MI behavior, indicating their potential as a valuable tool for enhancing MI proficiency in online helplines for mental health. Provided that the data set size is sufficiently large with enough training samples for each behavioral code, these methods can be trained and applied to other domains and languages, offering a scalable and cost-effective way to evaluate MI adherence, accelerate behavioral coding, and provide therapists with personalized, quick, and objective feedback.

Identifying populations at ultra-high risk of suicide using a novel machine learning method.

BACKGROUND: Targeted interventions for suicide prevention rely on adequate identification of groups at elevated risk. Several risk factors for suicide are known, but little is known about the interactions between risk factors. Interactions between risk factors may aid in detecting more specific sub-populations at higher risk. METHODS: Here, we use a novel machine learning heuristic to detect sub-populations at ultra high-risk for suicide based on interacting risk factors. The data-driven and hypothesis-free model is applied to investigate data covering the entire population of the Netherlands. FINDINGS: We found three sub-populations with extremely high suicide rates (i.e. >50 suicides per 100,000 person years, compared to 12/100,000 in the general population), namely: (1) people on unfit for work benefits that were never married, (2) males on unfit for work benefits, and (3) those aged 55-69 who live alone, were never married and have a relatively low household income. Additionally, we found two sub-populations where the rate was higher than expected based on individual risk factors alone: widowed males, and people aged 25-39 with a low level of education. INTERPRETATION: Our model is effective at finding ultra-high risk groups which can be targeted using sub-population level interventions. Additionally, it is effective at identifying high-risk groups that would not be considered risk groups based on conventional risk factor analysis.

Forecasting call and chat volumes at online helplines for mental health.

BACKGROUND: Each year, many help seekers in need contact health helplines for mental support. It is crucial that they receive support immediately, and that waiting times are minimal. In order to minimize delay, helplines must have adequate staffing levels, especially during peak hours. This has raised the need for means to predict the call and chat volumes ahead of time accurately. Motivated by this, in this paper, we analyze real-life data to develop models for accurately forecasting call volumes, for both phone and chat conversations for online mental health support. METHODS: This research was conducted on real call and chat data (adequately anonymized) provided by 113 Suicide Prevention (Over ons | 113 Zelfmoordpreventie) (throughout referred to as '113'), the online helpline for suicide prevention in the Netherlands. Chat and phone call data were analyzed to better understand the important factors that influence the call arrival process. These factors were then used as input to several Machine Learning (ML) models to forecast the number of call and chat arrivals. Next to that, senior counselors of the helpline completed a web-based questionnaire after each shift to assess their perception of the workload. RESULTS: This study has led to several remarkable and key insights. First, the most important factors that determine the call volumes for the helpline are the trend, and weekly and daily cyclic patterns (cycles), while monthly and yearly cycles were found to be non-significant predictors for the number of phone and chat conversations. Second, media events that were included in this study only have limited-and only short-term-impact on the call volumes. Third, so-called (S)ARIMA models are shown to lead to the most accurate prediction in the case of short-term forecasting, while simple linear models work best for long-term forecasting. Fourth, questionnaires filled in by senior counselors show that the experienced workload is mainly correlated to the number of chat conversations compared to phone calls. CONCLUSION: (S)ARIMA models can best be used to forecast the number of daily chats and phone calls with a MAPE of less than 10 in short-term forecasting. These models perform better than other models showing that the number of arrivals depends on historical data. These forecasts can be used as support for planning the number of counselors needed. Furthermore, the questionnaire data show that the workload experienced by senior counselors is more dependent on the number of chat arrivals and less on the number of available agents, showing the value of insight into the arrival process of conversations.

Detecting changes in help seeker conversations on a suicide prevention helpline during the COVID- 19 pandemic: in-depth analysis using encoder representations from transformers.

BACKGROUND: Preventatives measures to combat the spread of COVID- 19 have introduced social isolation, loneliness and financial stress. This study aims to identify whether the COVID-19 pandemic is related to changes in suicide-related problems for help seekers on a suicide prevention helpline. METHODS: A retrospective cohort study was conducted using chat data from a suicide prevention helpline in the Netherlands. The natural language processing method BERTopic was used to detect common topics in messages from December 1, 2019 until June 1, 2020 (N = 8589). Relative topic occurrence was compared before and during the lock down starting on March 23, 2020. The observed changes in topic usage were likewise analyzed for male and female, younger and older help seekers and help seekers living alone. RESULTS: The topic of the COVID-19 pandemic saw an 808% increase in relative occurrence after the lockdown. Furthermore, the results show that help seeker increased mention of thanking the counsellor (+ 15%), and male and young help seekers were grateful for the conversation (+ 45% and + 32% respectively). Coping methods such as watching TV (- 21%) or listening to music (- 15%) saw a decreased mention. Plans for suicide (- 9%) and plans for suicide at a specific location (- 15%) also saw a decreased mention. However, plans for suicide were mentioned more frequently by help seekers over 30 years old (+ 11%) or who live alone and (+ 52%). Furthermore, male help seekers talked about contact with emergency care (+ 43%) and panic and anxiety (+ 24%) more often. Negative emotions (+ 22%) and lack of self-confidence (+ 15%) were mentioned more often by help seekers under 30, and help seekers over 30 saw an increased mention of substance abuse (+ 9%). CONCLUSION: While mentions of distraction, social interaction and plans for suicide decreased, expressions of gratefulness for the helpline increased, highlighting the importance of contact to help seekers during the lockdown. Help seekers under 30, male or who live alone, showed changes that negatively related to suicidality and should be monitored closely.

Identifying socio-demographic risk factors for suicide using data on an individual level.

BACKGROUND: Suicide is a complex issue. Due to the relative rarity of the event, studies into risk factors are regularly limited by sample size or biased samples. The aims of the study were to find risk factors for suicide that are robust to intercorrelation, and which were based on a large and unbiased sample. METHODS: Using a training set of 5854 suicides and 596,416 control cases, we fit a logistic regression model and then evaluate the performance on a test set of 1425 suicides and 594,893 control cases. The data used was micro-data of Statistics Netherlands (CBS) with data on each inhabitant of the Netherlands. RESULTS: Taking the effect of possible correlating risk factors into account, those with a higher risk for suicide are men, middle-aged people, people with low income, those living alone, the unemployed, and those with mental or physical health problems. People with a lower risk are the highly educated, those with a non-western immigration background, and those living with a partner. CONCLUSION: We confirmed previously known risk factors such as male gender, middle-age, and low income and found that they are risk factors that are robust to intercorrelation. We found that debt and urbanicity were mostly insignificant and found that the regional differences found in raw frequencies are mostly explained away after correction of correlating risk factors, indicating that these differences were primarily caused due to the differences in the demographic makeup of the regions. We found an AUC of 0.77, which is high for a model predicting suicide death and comparable to the performance of deep learning models but with the benefit of remaining explainable.

The Most Effective Interventions for Classification Model Development to Predict Chat Outcomes Based on the Conversation Content in Online Suicide Prevention Chats: Machine Learning Approach.

BACKGROUND: For the provision of optimal care in a suicide prevention helpline, it is important to know what contributes to positive or negative effects on help seekers. Helplines can often be contacted through text-based chat services, which produce large amounts of text data for use in large-scale analysis. OBJECTIVE: We trained a machine learning classification model to predict chat outcomes based on the content of the chat conversations in suicide helplines and identified the counsellor utterances that had the most impact on its outputs. METHODS: From August 2021 until January 2023, help seekers (N=6903) scored themselves on factors known to be associated with suicidality (eg, hopelessness, feeling entrapped, will to live) before and after a chat conversation with the suicide prevention helpline in the Netherlands (113 Suicide Prevention). Machine learning text analysis was used to predict help seeker scores on these factors. Using 2 approaches for interpreting machine learning models, we identified text messages from helpers in a chat that contributed the most to the prediction of the model. RESULTS: According to the machine learning model, helpers' positive affirmations and expressing involvement contributed to improved scores of the help seekers. Use of macros and ending the chat prematurely due to the help seeker being in an unsafe situation had negative effects on help seekers. CONCLUSIONS: This study reveals insights for improving helpline chats, emphasizing the value of an evocative style with questions, positive affirmations, and practical advice. It also underscores the potential of machine learning in helpline chat analysis.

Avoiding Hospital Admissions and Delayed Transfers of Care by Improved Access to Intermediate Care: A Simulation Study.

OBJECTIVE: The current waiting times for intermediate care in the Netherlands prohibit timely access, leading to unwanted and costly hospital admissions. We propose alternative policies for improvement of intermediate care and estimate the effects on the waiting times, hospitalization, and the number of patient replacements. DESIGN: Simulation study. SETTING AND PARTICIPANTS: For our case study, data were used of older adults who received intermediate care in Amsterdam, the Netherlands, in 2019. For this target group, in- and outflows and patient characteristics were identified. METHODS: A process map of the main pathways into and out of the intermediate care was obtained and a discrete event simulation (DES) was built. We demonstrate the use of our DES for intermediate care by evaluating possible policy changes for a real-life case study in Amsterdam. RESULTS: By means of a sensitivity analysis with the DES, we show that in Amsterdam the waiting times are not a result of a lack in bed capacity but are due to an inefficient triage and application process. Older adults have to wait a median of 1.8 days for admission, leading to hospitalization. If the application process becomes more efficient and evening and weekend admissions are allowed, we find that unwanted hospitalization can be decreased substantially. CONCLUSION AND IMPLICATIONS: In this study, a simulation model is developed for intermediate care that can serve as a basis for policy decisions. Our case study shows that the waiting times for health care facilities are not always solved by increasing bed capacity. This underlines the necessity for a data-driven approach to identify logistic bottlenecks and find the best ways to solve them.

Reduced Waiting Times by Preference-Based Allocation of Patients to Nursing Homes.

OBJECTIVES: The long waiting times for nursing homes can be reduced by applying advanced waiting-line management. In this article, we implement a preference-based allocation model for older adults to nursing homes, evaluate the performance in a simulation setting for 2 case studies, and discuss the implementation in practice. DESIGN: Simulation study. SETTING AND PARTICIPANTS: Older adults requiring somatic nursing home care, from an urban region (Rotterdam) and a rural region (Twente) in the Netherlands. METHODS: Data about nursing homes and capacities for the 2 case studies were identified. A set of preference profiles was defined with aims regarding waiting time preferences and flexibility. Guidelines for implementation of the model in practice were obtained by addressing the tasks of all stakeholders. Thereafter, the simulation was run to compare the current practice with the allocation model based on specified outcome measures about waiting times and preferences. RESULTS: We found that the allocation model decreased the waiting times in both case studies. Compared with the current practice policy, the allocation model reduced the waiting times until placement by at least a factor of 2 (from 166 to 80 days in Rotterdam and 178 to 82 days in Twente). Moreover, more of the older adults ended up in their preferred nursing home and the aims of the distinct preference profiles were satisfied. CONCLUSIONS AND IMPLICATIONS: The results show that the allocation model outperforms commonly used waiting-line policies for nursing homes, while meeting individual preferences to a larger extent. Moreover, the model is easy to implement and of a generic nature and can, therefore, be extended to other settings as well (eg, to allocate older adults to home care or daycare). Finally, this research shows the potential of mathematical models in the care domain for older adults to face the increasing need for cost-effective solutions.

A Data-Driven Digital Application to Enhance the Capacity Planning of the COVID-19 Vaccination Process.

In this paper, a decision support system (DSS) is presented that focuses on the capacity planning of the COVID-19 vaccination process in the Netherlands. With the Dutch national vaccination priority list as the starting point, the DSS aims to minimize the per-class waiting-time with respect to (1) the locations of the medical hubs (i.e., the vaccination locations) and (2) the distribution of the available vaccines and healthcare professionals (over time). As the user is given the freedom to experiment with different starting positions and strategies, the DSS is ideally suited for providing support in the dynamic environment of the COVID-19 vaccination process. In addition to the DSS, a mathematical model to support the assignment of inhabitants to medical hubs is presented. This model has been satisfactorily implemented in practice in close collaboration with the Dutch Municipal and Regional Health Service (GGD GHOR Nederland).

Demographic Risk Factors for Suicide among Youths in The Netherlands.

In 2000 to 2016 the highest number of suicides among Dutch youths under 20 in any given year was 58 in 2013. In 2017 this number increased to 81 youth suicides. To get more insight in what types of youths died by suicide, particularly in recent years (2013-2017) we looked at micro-data of Statistics Netherlands and counted suicides among youths till 23, split out along gender, age, regions, immigration background and place in household and compared this to the general population of youths in the Netherlands. We also compared the demographics of young suicide victims to those of suicide victims among the population as a whole. We found higher suicide rates among male youths, older youths, those of Dutch descent and youths living alone. These differences were generally smaller than in the population as a whole. There were also substantial geographical differences between provinces and healthcare regions. The method of suicide is different in youth compared to the population as a whole: relatively more youth suicides by jumping or lying in front of a moving object and relatively less youth suicides by autointoxication or drowning, whereas the most frequent method of suicide among both groups is hanging or suffocation.

Can an Emoji a Day Keep the Doctor Away? An Explorative Mixed-Methods Feasibility Study to Develop a Self-Help App for Youth With Mental Health Problems.

Today's smartphones allow for a wide range of "big data" measurement, for example, ecological momentary assessment (EMA), whereby behaviours are repeatedly assessed within a person's natural environment. With this type of data, we can better understand - and predict - risk for behavioral and health issues and opportunities for (self-monitoring) interventions. In this mixed-methods feasibility study, through convenience sampling we collected data from 32 participants (aged 16-24) over a period of three months. To gain more insight into the app experiences of youth with mental health problems, we interviewed a subsample of 10 adolescents who received psycthological treatment. The results from this feasibility study indicate that emojis) can be used to identify positive and negative feelings, and individual pattern analyses of emojis may be useful for clinical purposes. While adolescents receiving mental health care are positive about future applications, these findings also highlight some caveats, such as possible drawback of inaccurate representation and incorrect predictions of emotional states. Therefore, at this stage, the app should always be combined with professional counseling. Results from this small pilot study warrant replication with studies of substantially larger sample size.