Predicting neuroticism with open-ended response using natural language processing.

Introduction: With rapid advancements in natural language processing (NLP), predicting personality using this technology has become a significant research interest. In personality prediction, exploring appropriate questions that elicit natural language is particularly important because questions determine the context of responses. This study aimed to predict levels of neuroticism-a core psychological trait known to predict various psychological outcomes-using responses to a series of open-ended questions developed based on the five-factor model of personality. This study examined the model's accuracy and explored the influence of item content in predicting neuroticism. Methods: A total of 425 Korean adults were recruited and responded to 18 open-ended questions about their personalities, along with the measurement of the Five-Factor Model traits. In total, 30,576 Korean sentences were collected. To develop the prediction models, the pre-trained language model KoBERT was used. Accuracy, F1 Score, Precision, and Recall were calculated as evaluation metrics. Results: The results showed that items inquiring about social comparison, unintended harm, and negative feelings performed better in predicting neuroticism than other items. For predicting depressivity, items related to negative feelings, social comparison, and emotions showed superior performance. For dependency, items related to unintended harm, social dominance, and negative feelings were the most predictive. Discussion: We identified items that performed better at neuroticism prediction than others. Prediction models developed based on open-ended questions that theoretically aligned with neuroticism exhibited superior predictive performance.

Final validation of the mental health screening tool for depressive disorders: A brief online and offline screening tool for major depressive disorder.

Early screening for depressive disorders is crucial given that major depressive disorder (MDD) is one of the main reasons of global burden of disease, and depression is the underlying cause for 60% of suicides. The need for an accurate screening for depression with high diagnostic sensitivity and specificity in a brief and culturally adapted manner has emerged. This study reports the final stage of a 3-year research project for the development of depression screening tool. The developed Mental Health Screening Tool for Depressive Disorders (MHS:D) was designed to be administered in both online and offline environments with a high level of sensitivity and specificity in screening for major depressive disorder. A total of 527 individuals completed two versions (online/offline) of the MHS:D and existing depression scales, including the BDI-II, CES-D, and PHQ-9. The Mini International Neuropsychiatric Interview (MINI) for diagnostic sensitivity/specificity was also administered to all participants. Internal consistency, convergent validity, factor analysis, item response theory analysis, and receiver operating characteristics curve (ROC) analysis were performed. The MHS:D showed an excellent level of internal consistency and convergent validity as well as a one-factor model with a reasonable level of model fit. The MHS:D could screen for major depressive disorder accurately (0.911 sensitivity and 0.878 specificity for both online and paper-pencil versions). Item response theory analysis suggested that items from the MHS:D could provide significantly more information than other existing depression scales. These statistical analyses indicated that the MHS:D is a valid and reliable scale for screening Korean patients with MDD with high diagnostic sensitivity and specificity. Moreover, given that the MHS:D is a considerably brief scale that can be administered in either online or paper-pencil versions, it can be used effectively in various contexts, particularly during the pandemic.

Predicting Personality and Psychological Distress Using Natural Language Processing: A Study Protocol.

Background: Self-report multiple choice questionnaires have been widely utilized to quantitatively measure one's personality and psychological constructs. Despite several strengths (e.g., brevity and utility), self-report multiple choice questionnaires have considerable limitations in nature. With the rise of machine learning (ML) and Natural language processing (NLP), researchers in the field of psychology are widely adopting NLP to assess psychological construct to predict human behaviors. However, there is a lack of connections between the work being performed in computer science and that of psychology due to small data sets and unvalidated modeling practices. Aims: The current article introduces the study method and procedure of phase II which includes the interview questions for the five-factor model (FFM) of personality developed in phase I. This study aims to develop the interview (semi-structured) and open-ended questions for the FFM-based personality assessments, specifically designed with experts in the field of clinical and personality psychology (phase 1), and to collect the personality-related text data using the interview questions and self-report measures on personality and psychological distress (phase 2). The purpose of the study includes examining the relationship between natural language data obtained from the interview questions, measuring the FFM personality constructs, and psychological distress to demonstrate the validity of the natural language-based personality prediction. Methods: Phase I (pilot) study was conducted to fifty-nine native Korean adults to acquire the personality-related text data from the interview (semi-structured) and open-ended questions based on the FFM of personality. The interview questions were revised and finalized with the feedback from the external expert committee, consisting of personality and clinical psychologists. Based on the established interview questions, a total of 300 Korean adults will be recruited using a convenience sampling method via online survey. The text data collected from interviews will be analyzed using the natural language processing. The results of the online survey including demographic data, depression, anxiety, and personality inventories will be analyzed together in the model to predict individuals' FFM of personality and the level of psychological distress (phase 2).

A meta-analysis of the effects of social-cognitive training in schizophrenia: The role of treatment characteristics and study quality.

OBJECTIVES: This meta-analysis was designed to assess the effects of social-cognitive training (SCT) and whether study quality, treatment approach, treatment context, and sample characteristics influence these effects. METHODS: Electronic databases were searched up to 5 August 2020 using variants of keywords: 'social cognition', 'training', 'rehabilitation', 'remediation', and 'schizophrenia'. Methodological moderators were extracted through the Clinical Trials Assessment Measure and verified by authors. This study was pre-registered on PROSPERO (CRD42020154026). RESULTS: Forty-two controlled trials with 1,868 participants were identified. The meta-analysis revealed moderate effects on emotion recognition, mental state attribution, and social perception. No significant effects were evident on psychiatric symptoms or social functioning. A small signal was evident for the generalization of treatment gains to executive function. Moderator analyses revealed that studies of lower methodological quality reported larger effects, and samples with lower mean years of education were associated with larger effects of SCT on mental state attribution. Treatment effects did not differ by other moderator variables such as treatment context and intervention types. CONCLUSIONS: SCT benefits people with schizophrenia on a variety of social-cognitive outcomes. Differences in baseline symptoms, gender distribution, antipsychotic medication dose, IQ, and other sample features did not create barriers to treatment benefits. Future studies should aim to enhance the generalization of training effects on broader clinical outcomes.

Reliability, and Convergent and Discriminant Validity of Gaming Disorder Scales: A Meta-Analysis.

Background: An association between gaming disorder (GD) and the symptoms of common mental disorders is unraveled yet. In this preregistered study, we quantitatively synthesized reliability, convergent and discriminant validity of GD scales to examine association between GD and other constructs. Methods: Five representative GD instruments (GAS-7, AICA, IGDT-10, Lemmens IGD-9, and IGDS9-SF) were chosen based on recommendations by the previous systematic review study to conduct correlation meta-analyses and reliability generalization. A systematic literature search was conducted through Pubmed, Proquest, Embase, and Google Scholar to identify studies that reported information on either reliability or correlation with related variables. 2,124 studies were full-text assessed as of October 2020, and 184 were quantitatively synthesized. Conventional Hedges two-level meta-analytic method was utilized. Results: The result of reliability generalization reported a mean coefficient alpha of 0.86 (95% CI = 0.85-0.87) and a mean test-retest estimate of 0.86 (95% CI = 0.81-0.89). Estimated effect sizes of correlation between GD and the variables were as follows: 0.33 with depression (k = 45; number of effect sizes), 0.29 with anxiety (k = 37), 0.30 with aggression (k = 19), -0.22 with quality of life (k = 18), 0.29 with loneliness (k = 18), 0.56 with internet addiction (k = 20), and 0.40 with game playtime (k = 53), respectively. The result of moderator analyses, funnel and forest plots, and publication bias analyses were also presented. Discussion and Conclusion: All five GD instruments have good internal consistency and test-retest reliability. Relatively few studies reported the test-retest reliability. The result of correlation meta-analysis revealed that GD scores were only moderately associated with game playtime. Common psychological problems such as depression and anxiety were found to have a slightly smaller association with GD than the gaming behavior. GD scores were strongly correlated with internet addiction. Further studies should adopt a rigorous methodological procedure to unravel the bidirectional relationship between GD and other psychopathologies. Limitations: The current study did not include gray literature. The representativeness of the five tools included in the current study could be questioned. High heterogeneity is another limitation of the study. Systematic Review Registration: [https://www.crd.york.ac.uk/PROSPERO/], identifier [CRD42020219781].

A Brief Online and Offline (Paper-and-Pencil) Screening Tool for Generalized Anxiety Disorder: The Final Phase in the Development and Validation of the Mental Health Screening Tool for Anxiety Disorders (MHS: A).

Generalized anxiety disorder (GAD) can cause significant socioeconomic burden and daily life dysfunction; hence, therapeutic intervention through early detection is important. This study was the final stage of a 3-year anxiety screening tool development project that evaluated the psychometric properties and diagnostic screening utility of the Mental Health Screening Tool for Anxiety Disorders (MHS: A), which measures GAD. A total of 527 Koreans completed online and offline (i.e., paper-and pencil) versions of the MHS: A, Beck Anxiety Inventory (BAI), Generalized Anxiety Disorder-7 (GAD-7), and Penn State Worry Questionnaire (PSWQ). The participants had an average age of 38.6 years and included 340 (64.5%) females. Participants were also administered the Mini-International Neuropsychiatric Interview (MINI). Internal consistency, convergent/criterion validity, item characteristics, and test information were assessed based on the item response theory (IRT), and a factor analysis and cut-off score analyses were conducted. The MHS: A had good internal consistency and good convergent validity with other anxiety scales. The two versions (online/offline) of the MHS: A were nearly identical (r = 0.908). It had a one-factor structure and showed better diagnostic accuracy (online/offline: sensitivity = 0.98/0.90, specificity = 0.80/0.83) for GAD detection than the GAD-7 and BAI. The IRT analysis indicated that the MHS: A was most informative as a screening tool for GAD. The MHS: A can serve as a clinically useful screening tool for GAD in Korea. Furthermore, it can be administered both online and offline and can be flexibly used as a brief mental health screener, especially with the current rise in telehealth.

The ultra brief checklist for suicidality.

BACKGROUND: Accurately identifying patients at an elevated risk of suicide in minimal time is an important mental health issue. Therefore, we developed the Ultra Brief Checklist for Suicidality (UBCS) and reported its psychometric properties. METHODS: We studied 544 participants who had completed the UBCS, other self-report measures, and structured clinical interviews by trained interviewers. We explored the psychometric properties of the UBCS's paper-based and online versions by identifying optimal cutoff points, sensitivity, specificity, etc. of all participants. RESULTS: The mean score of the UBCS (paper-pencil version) in the no-suicide-risk group was 0.23 (SD = 0.73), while in the Risk-Positive (RP) and the High-Risk (HR) groups it was 3.54 (SD = 3.22) and 6.31 (SD = 3.97), respectively. Optimal cutoff points for the RP and HR groups were 1 (AUC = 0.914) and 3 (AUC = 0.928), respectively. LIMITATIONS: The current findings apply only to adults, not to children or adolescents. The original MINI was used as the assessment instrument's reference standard rather than MINI 7.0.2 of DSM-5. Predictive validity was not addressed as this was not a longitudinal cohort study. CONCLUSIONS: The UBCS was rigorously tested and validated. More than half the participants completed the UBCS within a minute. It possesses adequate psychometric properties and can be easily administered with minimal clinician time and effort, via paper or Internet. We believe it can contribute to suicide risk identification and timely interventions.

Diagnostic Utility and Psychometric Properties of the Beck Depression Inventory-II Among Korean Adults.

The Beck Depression Inventory-II (BDI-II) is one of the most widely used depression assessment tools in Korea. However, the psychometric properties and diagnostic cut-off point of the official Korean version of the BDI-II have not yet been reported. This study aims to clarify the psychometric properties and diagnostic utility of the Korean BDI-II. A total of 1,145 clinical and non-clinical Korean adults participated in this study. The BDI-II showed a high level of internal consistency and high correlations with other depression-related measures. Confirmatory factor analysis (CFA) was performed, and a 3-factor model showed the best model fit. To identify the diagnostic utility of the BDI-II, the Quality Assessment of Diagnostic Accuracy Studies 2nd Edition (QUADAS-2) methodology was applied in participant recruitment and research design. Results of ROC curve analysis suggested two optimal cut-off scores, 23 points for detecting major depressive disorder (MDD) (83.3% sensitivity, 86.8% specificity) and 17 points for depressive-related disorder (80.9% sensitivity, 76.4% specificity). To identify the usefulness of the BDI-II as a severity assessment tool or screening tool, a test information curve (TIC) was generated with an Item Response Theory (IRT) analysis. The TIC was flat and plateau-like, indicating its appropriateness as a severity rating tool. Research data supports the BDI-II as a reliable and valid screening tool as well as a severity rating tool in the Korean adult population.

Clinical Utility of Beck Anxiety Inventory in Clinical and Nonclinical Korean Samples.

Despite the prominent use of the Beck Anxiety Inventory (BAI) in primary healthcare systems, few studies have confirmed its diagnostic utility and psychometric properties in non-Western countries. This study aims to clarify the clinical utility of the BAI as a screening tool for anxiety disorders according to DSM-IV criteria, based on blind recruitment and diagnostic interviews of both clinical and non-clinical participants in the Korean population. A total of 1,157 participants were involved in the final psychometric analysis, which included correlational analysis with other anxiety and depression self-report measures and mean score comparison with the Beck Depression Inventory (BDI). ROC analysis and calculation of positive and negative predictive values were conducted to examine diagnostic utility. The BAI was found to have high correlations with depression-related self-report measures (0.747-0.796) and moderate to high correlations with anxiety-related self-report measures (0.518-0.776). The ROC analysis failed to provide cutoff scores with adequate sensitivity and specificity for identifying participants with anxiety disorders (85.0% sensitivity, 88.1% specificity, and 92.8% AUC). The comparison of BAI and BDI mean scores for different diagnostic groups revealed that BAI and BDI scores were higher in the depressive or anxiety disorders group than in the non-clinical group. However, BAI mean score was not higher for the anxiety-only group than the depression-only group. Our data supports the BAI reliability and validity as a tool to measure the severity of general anxiety in clinical and non-clinical populations; however, it fails to capture the unique characteristics of anxiety disorders that distinguish them from depressive disorders. Further clinical implications of the BAI based on these results and some limitations of the study are discussed.