Comparative effectiveness of remote digital gamified and group CBT skills training interventions for anxiety and depression among college students: Results of a three-arm randomised controlled trial.

Digital interventions can enhance access to healthcare in under-resourced settings. However, guided digital interventions may be costly for low- and middle-income countries, despite their effectiveness. In this randomised control trial, we evaluated the effectiveness of two digital interventions designed to address this issue: (1) a Cognitive Behavioral Therapy Skills Training (CST) intervention that increased scalability by using remote online group administration; and (2) the SuperBetter gamified self-guided CBT skills training app, which uses other participants rather than paid staff as guides. The study was implemented among anxious and/or depressed South African undergraduates (n = 371) randomised with equal allocation to Remote Group CST, SuperBetter, or a MoodFlow mood monitoring control. Symptoms were assessed with the Generalized Anxiety Disorder-7 (GAD-7) and the Patient Health Questionnaire-9 (PHQ-9). Intention-to-treat analysis found effect sizes at the high end of prior digital intervention trials, including significantly higher adjusted risk differences (ARD; primary outcome) in joint anxiety/depression remission at 3-months and 6-months for Remote Group CST (ARD = 23.3-18.9%, p = 0.001-0.035) and SuperBetter (ARD = 12.7-22.2%, p = 0.047-0.006) than MoodFlow and mean combined PHQ-9/GAD-7 scores (secondary outcome) significantly lower for Remote Group CST and SuperBetter than MoodFlow. These results illustrate how innovative delivery methods can increase the scalability of standard one-on-one guided digital interventions. PREREGISTRATION INTERNATIONAL STANDARD RANDOMISED CONTROLLED TRIAL NUMBER (ISRTCN) SUBMISSION #: 47,089,643.

Developing an individualized treatment rule for Veterans with major depressive disorder using electronic health records.

Efforts to develop an individualized treatment rule (ITR) to optimize major depressive disorder (MDD) treatment with antidepressant medication (ADM), psychotherapy, or combined ADM-psychotherapy have been hampered by small samples, small predictor sets, and suboptimal analysis methods. Analyses of large administrative databases designed to approximate experiments followed iteratively by pragmatic trials hold promise for resolving these problems. The current report presents a proof-of-concept study using electronic health records (EHR) of n = 43,470 outpatients beginning MDD treatment in Veterans Health Administration Primary Care Mental Health Integration (PC-MHI) clinics, which offer access not only to ADMs but also psychotherapy and combined ADM-psychotherapy. EHR and geospatial databases were used to generate an extensive baseline predictor set (5,865 variables). The outcome was a composite measure of at least one serious negative event (suicide attempt, psychiatric emergency department visit, psychiatric hospitalization, suicide death) over the next 12 months. Best-practices methods were used to adjust for nonrandom treatment assignment and to estimate a preliminary ITR in a 70% training sample and to evaluate the ITR in the 30% test sample. Statistically significant aggregate variation was found in overall probability of the outcome related to baseline predictors (AU-ROC = 0.68, S.E. = 0.01), with test sample outcome prevalence of 32.6% among the 5% of patients having highest predicted risk compared to 7.1% in the remainder of the test sample. The ITR found that psychotherapy-only was the optimal treatment for 56.0% of patients (roughly 20% lower risk of the outcome than if receiving one of the other treatments) and that treatment type was unrelated to outcome risk among other patients. Change in aggregate treatment costs of implementing this ITR would be negligible, as 16.1% fewer patients would be prescribed ADMs and 2.9% more would receive psychotherapy. A pragmatic trial would be needed to confirm the accuracy of the ITR.

Predicting Homelessness Among Transitioning U.S. Army Soldiers.

INTRODUCTION: This study develops a practical method to triage Army transitioning service members (TSMs) at highest risk of homelessness to target a preventive intervention. METHODS: The sample included 4,790 soldiers from the Study to Assess Risk and Resilience in Servicemembers-Longitudinal Study (STARRS-LS) who participated in 1 of 3 Army STARRS 2011-2014 baseline surveys followed by the third wave of the STARRS-LS online panel surveys (2020-2022). Two machine learning models were trained: a Stage-1 model that used administrative predictors and geospatial data available for all TSMs at discharge to identify high-risk TSMs for initial outreach; and a Stage-2 model estimated in the high-risk subsample that used self-reported survey data to help determine highest risk based on additional information collected from high-risk TSMs once they are contacted. The outcome in both models was homelessness within 12 months after leaving active service. RESULTS: Twelve-month prevalence of post-transition homelessness was 5.0% (SE=0.5). The Stage-1 model identified 30% of high-risk TSMs who accounted for 52% of homelessness. The Stage-2 model identified 10% of all TSMs (i.e., 33% of high-risk TSMs) who accounted for 35% of all homelessness (i.e., 63% of the homeless among high-risk TSMs). CONCLUSIONS: Machine learning can help target outreach and assessment of TSMs for homeless prevention interventions.

Estimated Average Treatment Effect of Psychiatric Hospitalization in Patients With Suicidal Behaviors: A Precision Treatment Analysis.

Importance: Psychiatric hospitalization is the standard of care for patients presenting to an emergency department (ED) or urgent care (UC) with high suicide risk. However, the effect of hospitalization in reducing subsequent suicidal behaviors is poorly understood and likely heterogeneous. Objectives: To estimate the association of psychiatric hospitalization with subsequent suicidal behaviors using observational data and develop a preliminary predictive analytics individualized treatment rule accounting for heterogeneity in this association across patients. Design, Setting, and Participants: A machine learning analysis of retrospective data was conducted. All veterans presenting with suicidal ideation (SI) or suicide attempt (SA) from January 1, 2010, to December 31, 2015, were included. Data were analyzed from September 1, 2022, to March 10, 2023. Subgroups were defined by primary psychiatric diagnosis (nonaffective psychosis, bipolar disorder, major depressive disorder, and other) and suicidality (SI only, SA in past 2-7 days, and SA in past day). Models were trained in 70.0% of the training samples and tested in the remaining 30.0%. Exposures: Psychiatric hospitalization vs nonhospitalization. Main Outcomes and Measures: Fatal and nonfatal SAs within 12 months of ED/UC visits were identified in administrative records and the National Death Index. Baseline covariates were drawn from electronic health records and geospatial databases. Results: Of 196 610 visits (90.3% men; median [IQR] age, 53 [41-59] years), 71.5% resulted in hospitalization. The 12-month SA risk was 11.9% with hospitalization and 12.0% with nonhospitalization (difference, -0.1%; 95% CI, -0.4% to 0.2%). In patients with SI only or SA in the past 2 to 7 days, most hospitalization was not associated with subsequent SAs. For patients with SA in the past day, hospitalization was associated with risk reductions ranging from -6.9% to -9.6% across diagnoses. Accounting for heterogeneity, hospitalization was associated with reduced risk of subsequent SAs in 28.1% of the patients and increased risk in 24.0%. An individualized treatment rule based on these associations may reduce SAs by 16.0% and hospitalizations by 13.0% compared with current rates. Conclusions and Relevance: The findings of this study suggest that psychiatric hospitalization is associated with reduced average SA risk in the immediate aftermath of an SA but not after other recent SAs or SI only. Substantial heterogeneity exists in these associations across patients. An individualized treatment rule accounting for this heterogeneity could both reduce SAs and avert hospitalizations.

Internet-delivered cognitive behavior therapy versus treatment as usual for anxiety and depression among Latin American university students: A randomized clinical trial.

OBJECTIVE: Untreated mental disorders are important among low- and middle-income country (LMIC) university students in Latin America, where barriers to treatment are high. Scalable interventions are needed. This study compared transdiagnostic self-guided and guided internet-delivered cognitive behavioral therapy (i-CBT) with treatment as usual (TAU) for clinically significant anxiety and depression among undergraduates in Colombia and Mexico. METHOD: 1,319 anxious, as determined by the Generalized Anxiety Disorder-7 (GAD-7) = 10+ and/or depressed, as determined by the Patient Health Questionnaire-9 (PHQ-9) = 10+, undergraduates (mean [SD] age = 21.4 [3.2]); 78.7% female; 55.9% first-generation university student) from seven universities in Colombia and Mexico were randomized to culturally adapted versions of self-guided i-CBT (n = 439), guided i-CBT (n = 445), or treatment as usual (TAU; n = 435). All randomized participants were reassessed 3 months after randomization. The primary outcome was remission of both anxiety (GAD-7 = 0-4) and depression (PHQ-9 = 0-4). We hypothesized that remission would be higher with guided i-CBT than with the other interventions. RESULTS: Intent-to-treat analysis found significantly higher adjusted (for university and loss to follow-up) remission rates (ARD) among participants randomized to guided i-CBT than either self-guided i-CBT (ARD = 13.1%, χ12 = 10.4, p = .001) or TAU (ARD = 11.2%, χ12 = 8.4, p = .004), but no significant difference between self-guided i-CBT and TAU (ARD = -1.9%, χ12 = 0.2, p = .63). Per-protocol sensitivity analyses and analyses of dimensional outcomes yielded similar results. CONCLUSIONS: Significant reductions in anxiety and depression among LMIC university students could be achieved with guided i-CBT, although further research is needed to determine which students would most likely benefit from this intervention. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

A practical risk calculator for suicidal behavior among transitioning U.S. Army soldiers: results from the Study to Assess Risk and Resilience in Servicemembers-Longitudinal Study (STARRS-LS).

BACKGROUND: Risk of suicide-related behaviors is elevated among military personnel transitioning to civilian life. An earlier report showed that high-risk U.S. Army soldiers could be identified shortly before this transition with a machine learning model that included predictors from administrative systems, self-report surveys, and geospatial data. Based on this result, a Veterans Affairs and Army initiative was launched to evaluate a suicide-prevention intervention for high-risk transitioning soldiers. To make targeting practical, though, a streamlined model and risk calculator were needed that used only a short series of self-report survey questions. METHODS: We revised the original model in a sample of n = 8335 observations from the Study to Assess Risk and Resilience in Servicemembers-Longitudinal Study (STARRS-LS) who participated in one of three Army STARRS 2011-2014 baseline surveys while in service and in one or more subsequent panel surveys (LS1: 2016-2018, LS2: 2018-2019) after leaving service. We trained ensemble machine learning models with constrained numbers of item-level survey predictors in a 70% training sample. The outcome was self-reported post-transition suicide attempts (SA). The models were validated in the 30% test sample. RESULTS: Twelve-month post-transition SA prevalence was 1.0% (s.e. = 0.1). The best constrained model, with only 17 predictors, had a test sample ROC-AUC of 0.85 (s.e. = 0.03). The 10-30% of respondents with the highest predicted risk included 44.9-92.5% of 12-month SAs. CONCLUSIONS: An accurate SA risk calculator based on a short self-report survey can target transitioning soldiers shortly before leaving service for intervention to prevent post-transition SA.

Findings From the World Mental Health Surveys of Civil Violence Exposure and Its Association With Subsequent Onset and Persistence of Mental Disorders.

Importance: Understanding the association of civil violence with mental disorders is important for developing effective postconflict recovery policies. Objective: To estimate the association between exposure to civil violence and the subsequent onset and persistence of common mental disorders (in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition [DSM-IV]) in representative surveys of civilians from countries that have experienced civil violence since World War II. Design, Setting, and Participants: This study used data from cross-sectional World Health Organization World Mental Health (WMH) surveys administered to households between February 5, 2001, and January 5, 2022, in 7 countries that experienced periods of civil violence after World War II (Argentina, Colombia, Lebanon, Nigeria, Northern Ireland, Peru, and South Africa). Data from respondents in other WMH surveys who immigrated from countries with civil violence in Africa and Latin America were also included. Representative samples comprised adults (aged ≥18 years) from eligible countries. Data analysis was performed from February 10 to 13, 2023. Exposures: Exposure was defined as a self-report of having been a civilian in a war zone or region of terror. Related stressors (being displaced, witnessing atrocities, or being a combatant) were also assessed. Exposures occurred a median of 21 (IQR, 12-30) years before the interview. Main Outcomes and Measures: The main outcome was the retrospectively reported lifetime prevalence and 12-month persistence (estimated by calculating 12-month prevalence among lifetime cases) of DSM-IV anxiety, mood, and externalizing (alcohol use, illicit drug use, or intermittent explosive) disorders. Results: This study included 18 212 respondents from 7 countries. Of these individuals, 2096 reported that they were exposed to civil violence (56.5% were men; median age, 40 [IQR, 30-52] years) and 16 116 were not exposed (45.2% were men; median age, 35 [IQR, 26-48] years). Respondents who reported being exposed to civil violence had a significantly elevated onset risk of anxiety (risk ratio [RR], 1.8 [95% CI, 1.5-2.1]), mood (RR, 1.5 [95% CI, 1.3-1.7]), and externalizing (RR, 1.6 [95% CI, 1.3-1.9]) disorders. Combatants additionally had a significantly elevated onset risk of anxiety disorders (RR, 2.0 [95% CI, 1.3-3.1]) and refugees had an increased onset risk of mood (RR, 1.5 [95% CI, 1.1-2.0]) and externalizing (RR, 1.6 [95% CI, 1.0-2.4]) disorders. Elevated disorder onset risks persisted for more than 2 decades if conflicts persisted but not after either termination of hostilities or emigration. Persistence (ie, 12-month prevalence among respondents with lifetime prevalence of the disorder), in comparison, was generally not associated with exposure. Conclusions: In this survey study of exposure to civil violence, exposure was associated with an elevated risk of mental disorders among civilians for many years after initial exposure. These findings suggest that policy makers should recognize these associations when projecting future mental disorder treatment needs in countries experiencing civil violence and among affected migrants.

A Precision Treatment Model for Internet-Delivered Cognitive Behavioral Therapy for Anxiety and Depression Among University Students: A Secondary Analysis of a Randomized Clinical Trial.

Importance: Guided internet-delivered cognitive behavioral therapy (i-CBT) is a low-cost way to address high unmet need for anxiety and depression treatment. Scalability could be increased if some patients were helped as much by self-guided i-CBT as guided i-CBT. Objective: To develop an individualized treatment rule using machine learning methods for guided i-CBT vs self-guided i-CBT based on a rich set of baseline predictors. Design, Setting, and Participants: This prespecified secondary analysis of an assessor-blinded, multisite randomized clinical trial of guided i-CBT, self-guided i-CBT, and treatment as usual included students in Colombia and Mexico who were seeking treatment for anxiety (defined as a 7-item Generalized Anxiety Disorder [GAD-7] score of ≥10) and/or depression (defined as a 9-item Patient Health Questionnaire [PHQ-9] score of ≥10). Study recruitment was from March 1 to October 26, 2021. Initial data analysis was conducted from May 23 to October 26, 2022. Interventions: Participants were randomized to a culturally adapted transdiagnostic i-CBT that was guided (n = 445), self-guided (n = 439), or treatment as usual (n = 435). Main Outcomes and Measures: Remission of anxiety (GAD-7 scores of ≤4) and depression (PHQ-9 scores of ≤4) 3 months after baseline. Results: The study included 1319 participants (mean [SD] age, 21.4 [3.2] years; 1038 women [78.7%]; 725 participants [55.0%] came from Mexico). A total of 1210 participants (91.7%) had significantly higher mean (SE) probabilities of joint remission of anxiety and depression with guided i-CBT (51.8% [3.0%]) than with self-guided i-CBT (37.8% [3.0%]; P = .003) or treatment as usual (40.0% [2.7%]; P = .001). The remaining 109 participants (8.3%) had low mean (SE) probabilities of joint remission of anxiety and depression across all groups (guided i-CBT: 24.5% [9.1%]; P = .007; self-guided i-CBT: 25.4% [8.8%]; P = .004; treatment as usual: 31.0% [9.4%]; P = .001). All participants with baseline anxiety had nonsignificantly higher mean (SE) probabilities of anxiety remission with guided i-CBT (62.7% [5.9%]) than the other 2 groups (self-guided i-CBT: 50.2% [6.2%]; P = .14; treatment as usual: 53.0% [6.0%]; P = .25). A total of 841 of 1177 participants (71.5%) with baseline depression had significantly higher mean (SE) probabilities of depression remission with guided i-CBT (61.5% [3.6%]) than the other 2 groups (self-guided i-CBT: 44.3% [3.7%]; P = .001; treatment as usual: 41.8% [3.2%]; P < .001). The other 336 participants (28.5%) with baseline depression had nonsignificantly higher mean (SE) probabilities of depression remission with self-guided i-CBT (54.4% [6.0%]) than guided i-CBT (39.8% [5.4%]; P = .07). Conclusions and Relevance: Guided i-CBT yielded the highest probabilities of remission of anxiety and depression for most participants; however, these differences were nonsignificant for anxiety. Some participants had the highest probabilities of remission of depression with self-guided i-CBT. Information about this variation could be used to optimize allocation of guided and self-guided i-CBT in resource-constrained settings. Trial Registration: ClinicalTrials.gov Identifier: NCT04780542.

Evaluation of a Model to Target High-risk Psychiatric Inpatients for an Intensive Postdischarge Suicide Prevention Intervention.

Importance: The months after psychiatric hospital discharge are a time of high risk for suicide. Intensive postdischarge case management, although potentially effective in suicide prevention, is likely to be cost-effective only if targeted at high-risk patients. A previously developed machine learning (ML) model showed that postdischarge suicides can be predicted from electronic health records and geospatial data, but it is unknown if prediction could be improved by adding additional information. Objective: To determine whether model prediction could be improved by adding information extracted from clinical notes and public records. Design, Setting, and Participants: Models were trained to predict suicides in the 12 months after Veterans Health Administration (VHA) short-term (less than 365 days) psychiatric hospitalizations between the beginning of 2010 and September 1, 2012 (299 050 hospitalizations, with 916 hospitalizations followed within 12 months by suicides) and tested in the hospitalizations from September 2, 2012, to December 31, 2013 (149 738 hospitalizations, with 393 hospitalizations followed within 12 months by suicides). Validation focused on net benefit across a range of plausible decision thresholds. Predictor importance was assessed with Shapley additive explanations (SHAP) values. Data were analyzed from January to August 2022. Main Outcomes and Measures: Suicides were defined by the National Death Index. Base model predictors included VHA electronic health records and patient residential data. The expanded predictors came from natural language processing (NLP) of clinical notes and a social determinants of health (SDOH) public records database. Results: The model included 448 788 unique hospitalizations. Net benefit over risk horizons between 3 and 12 months was generally highest for the model that included both NLP and SDOH predictors (area under the receiver operating characteristic curve range, 0.747-0.780; area under the precision recall curve relative to the suicide rate range, 3.87-5.75). NLP and SDOH predictors also had the highest predictor class-level SHAP values (proportional SHAP = 64.0% and 49.3%, respectively), although the single highest positive variable-level SHAP value was for a count of medications classified by the US Food and Drug Administration as increasing suicide risk prescribed the year before hospitalization (proportional SHAP = 15.0%). Conclusions and Relevance: In this study, clinical notes and public records were found to improve ML model prediction of suicide after psychiatric hospitalization. The model had positive net benefit over 3-month to 12-month risk horizons for plausible decision thresholds. Although caution is needed in inferring causality based on predictor importance, several key predictors have potential intervention implications that should be investigated in future studies.

Estimated Prevalence of and Factors Associated With Clinically Significant Anxiety and Depression Among US Adults During the First Year of the COVID-19 Pandemic.

Importance: Claims of dramatic increases in clinically significant anxiety and depression early in the COVID-19 pandemic came from online surveys with extremely low or unreported response rates. Objective: To examine trend data in a calibrated screening for clinically significant anxiety and depression among adults in the only US government benchmark probability trend survey not disrupted by the COVID-19 pandemic. Design, Setting, and Participants: This survey study used the US Centers for Disease Control and Prevention Behavioral Risk Factor Surveillance System (BRFSS), a monthly state-based trend survey conducted over the telephone. Participants were adult respondents in the 50 US states and District of Columbia who were surveyed March to December 2020 compared with the same months in 2017 to 2019. Exposures: Monthly state COVID-19 death rates. Main Outcomes and Measures: Estimated 30-day prevalence of clinically significant anxiety and depression based on responses to a single BRFSS item calibrated to a score of 6 or greater on the 4-item Patient Health Questionnaire (area under the receiver operating characteristic curve, 0.84). All percentages are weighted based on BRFSS calibration weights. Results: Overall, there were 1 429 354 respondents, with 1 093 663 in 2017 to 2019 (600 416 [51.1%] women; 87 153 [11.8%] non-Hispanic Black; 826 334 [61.5%] non-Hispanic White; 411 254 [27.8%] with college education; and 543 619 [56.8] employed) and 335 691 in 2020 (182 351 [51.3%] women; 25 517 [11.7%] non-Hispanic Black; 250 333 [60.5%] non-Hispanic White; 130 642 [29.3%] with college education; and 168 921 [54.9%] employed). Median within-state response rates were 45.9% to 49.4% in 2017 to 2019 and 47.9% in 2020. Estimated 30-day prevalence of clinically significant anxiety and depression was 0.4 (95% CI, 0.0 to 0.7) percentage points higher in March to December 2020 (12.4%) than March to December 2017 to 2019 (12.1%). This estimated increase was limited, however, to students (2.4 [95% CI, 0.8 to 3.9] percentage points) and the employed (0.9 [95% CI, 0.5 to 1.4] percentage points). Estimated prevalence decreased among the short-term unemployed (-1.8 [95% CI, -3.1 to -0.5] percentage points) and those unable to work (-4.2 [95% CI, -5.3 to -3.2] percentage points), but did not change significantly among the long-term unemployed (-2.1 [95% CI, -4.5 to 0.5] percentage points), homemakers (0.8 [95% CI, -0.3 to 1.9] percentage points), or the retired (0.1 [95% CI, -0.6 to 0.8] percentage points). The increase in anxiety and depression prevalence among employed people was positively associated with the state-month COVID-19 death rate (1.8 [95% CI, 1.2 to 2.5] percentage points when high and 0.0 [95% CI, -0.7 to 0.6] percentage points when low) and was elevated among women compared with men (2.0 [95% CI, 1.4 to 2.5] percentage points vs 0.2 [95% CI, -0.1 to 0.6] percentage points), Non-Hispanic White individuals compared with Hispanic and non-Hispanic Black individuals (1.3 [95% CI, 0.6 to 1.9] percentage points vs 1.1 [95% CI, -0.2 to 2.5] percentage points and 0.7 [95% CI, -0.1 to 1.5] percentage points), and those with college educations compared with less than high school educations (2.5 [95% CI, 1.9 to 3.1] percentage points vs -0.6 [95% CI, -2.7 to 1.4] percentage points). Conclusions and Relevance: In this survey study, clinically significant US adult anxiety and depression increased less during 2020 than suggested by online surveys. However, this modest aggregate increase could mask more substantial increases in key population segments (eg, first responders) and might have become larger in 2021 and 2022.

Predicting Homelessness Among U.S. Army Soldiers No Longer on Active Duty.

INTRODUCTION: The ability to predict and prevent homelessness has been an elusive goal. The purpose of this study was to develop a prediction model that identified U.S. Army soldiers at high risk of becoming homeless after transitioning to civilian life based on information available before the time of this transition. METHODS: The prospective cohort study consisted of observations from 16,589 soldiers who were separated or deactivated from service and who had previously participated in 1 of 3 baseline surveys of the Army Study to Assess Risk and Resilience in Servicemembers in 2011-2014. A machine learning model was developed in a 70% training sample and evaluated in the remaining 30% test sample to predict self-reported homelessness in 1 of 2 Longitudinal Study surveys administered in 2016-2018 and 2018-2019. Predictors included survey, administrative, and geospatial variables available before separation/deactivation. Analysis was conducted in November 2020-May 2021. RESULTS: The 12-month prevalence of homelessness was 2.9% (SE=0.2%) in the total Longitudinal Study sample. The area under the receiver operating characteristic curve in the test sample was 0.78 (SE=0.02) for homelessness. The 4 highest ventiles (top 20%) of predicted risk included 61% of respondents with homelessness. Self-reported lifetime histories of depression, trauma of having a loved one murdered, and post-traumatic stress disorder were the 3 strongest predictors of homelessness. CONCLUSIONS: A prediction model for homelessness can accurately target soldiers for preventive intervention before transition to civilian life.

Prediction of Suicide Attempts Using Clinician Assessment, Patient Self-report, and Electronic Health Records.

Importance: Half of the people who die by suicide make a health care visit within 1 month of their death. However, clinicians lack the tools to identify these patients. Objective: To predict suicide attempts within 1 and 6 months of presentation at an emergency department (ED) for psychiatric problems. Design, Setting, and Participants: This prognostic study assessed the 1-month and 6-month risk of suicide attempts among 1818 patients presenting to an ED between February 4, 2015, and March 13, 2017, with psychiatric problems. Data analysis was performed from May 1, 2020, to November 19, 2021. Main Outcomes and Measures: Suicide attempts 1 and 6 months after presentation to the ED were defined by combining data from electronic health records (EHRs) with patient 1-month (n = 1102) and 6-month (n = 1220) follow-up surveys. Ensemble machine learning was used to develop predictive models and a risk score for suicide. Results: A total of 1818 patients participated in this study (1016 men [55.9%]; median age, 33 years [IQR, 24-46 years]; 266 Hispanic patients [14.6%]; 1221 non-Hispanic White patients [67.2%], 142 non-Hispanic Black patients [7.8%], 64 non-Hispanic Asian patients [3.5%], and 125 non-Hispanic patients of other race and ethnicity [6.9%]). A total of 137 of 1102 patients (12.9%; weighted prevalence) attempted suicide within 1 month, and a total of 268 of 1220 patients (22.0%; weighted prevalence) attempted suicide within 6 months. Clinicians' assessment alone was little better than chance at predicting suicide attempts, with externally validated area under the receiver operating characteristic curve (AUC) of 0.67 for the 1-month model and 0.60 for the 6-month model. Prediction accuracy was slightly higher for models based on EHR data (1-month model: AUC, 0.71; 6 month model: AUC, 0.65) and was best using patient self-reports (1-month model: AUC, 0.76; 6-month model: AUC, 0.77), especially when patient self-reports were combined with EHR and/or clinician data (1-month model: AUC, 0.77; and 6 month model: AUC, 0.79). A model that used only 20 patient self-report questions and an EHR-based risk score performed similarly well (1-month model: AUC, 0.77; 6 month model: AUC, 0.78). In the best 1-month model, 30.7% (positive predicted value) of the patients classified as having highest risk (top 25% of the sample) made a suicide attempt within 1 month of their ED visit, accounting for 64.8% (sensitivity) of all 1-month attempts. In the best 6-month model, 46.0% (positive predicted value) of the patients classified at highest risk made a suicide attempt within 6 months of their ED visit, accounting for 50.2% (sensitivity) of all 6-month attempts. Conclusions and Relevance: This prognostic study suggests that the ability to identify patients at high risk of suicide attempt after an ED visit for psychiatric problems improved using a combination of patient self-reports and EHR data.

Predicting suicide attempts among U.S. Army soldiers after leaving active duty using information available before leaving active duty: results from the Study to Assess Risk and Resilience in Servicemembers-Longitudinal Study (STARRS-LS).

Suicide risk is elevated among military service members who recently transitioned to civilian life. Identifying high-risk service members before this transition could facilitate provision of targeted preventive interventions. We investigated the feasibility of doing this by attempting to develop a prediction model for self-reported suicide attempts (SAs) after leaving or being released from active duty in the Study to Assess Risk and Resilience in Servicemembers-Longitudinal Study (STARRS-LS). This study included two self-report panel surveys (LS1: 2016-2018, LS2: 2018-2019) administered to respondents who previously participated while on active duty in one of three Army STARRS 2011-2014 baseline self-report surveys. We focus on respondents who left active duty >12 months before their LS survey (n = 8899). An ensemble machine learning model using predictors available prior to leaving active duty was developed in a 70% training sample and validated in a 30% test sample. The 12-month self-reported SA prevalence (SE) was 1.0% (0.1). Test sample AUC (SE) was 0.74 (0.06). The 15% of respondents with highest predicted risk included nearly two-thirds of 12-month SAs and over 80% of medically serious 12-month SAs. These results show that it is possible to identify soldiers at high post-transition self-report SA risk before the transition. Future model development is needed to examine prediction of SAs assessed by administrative data and using surveys administered closer to the time of leaving active duty.

Evaluating the heterogeneous effect of a modifiable risk factor on suicide: The case of vitamin D deficiency.

OBJECTIVES: To illustrate the use of machine learning methods to search for heterogeneous effects of a target modifiable risk factor on suicide in observational studies. The illustration focuses on secondary analysis of a matched case-control study of vitamin D deficiency predicting subsequent suicide. METHODS: We describe a variety of machine learning methods to search for prescriptive predictors; that is, predictors of significant variation in the association between a target risk factor and subsequent suicide. In each case, the purpose is to evaluate the potential value of selective intervention on the target risk factor to prevent the outcome based on the provisional assumption that the target risk factor is causal. The approaches illustrated include risk modeling based on the super learner ensemble machine learning method, Least Absolute Shrinkage and Selection Operator (Lasso) penalized regression, and the causal forest algorithm. RESULTS: The logic of estimating heterogeneous intervention effects is exposited along with the illustration of some widely used methods for implementing this logic. CONCLUSIONS: In addition to describing best practices in using the machine learning methods considered here, we close with a discussion of broader design and analysis issues in planning an observational study to investigate heterogeneous effects of a modifiable risk factor.

Development and Validation of a Model to Predict Posttraumatic Stress Disorder and Major Depression After a Motor Vehicle Collision.

Importance: A substantial proportion of the 40 million people in the US who present to emergency departments (EDs) each year after traumatic events develop posttraumatic stress disorder (PTSD) or major depressive episode (MDE). Accurately identifying patients at high risk in the ED would facilitate the targeting of preventive interventions. Objectives: To develop and validate a prediction tool based on ED reports after a motor vehicle collision to predict PTSD or MDE 3 months later. Design, Setting, and Participants: The Advancing Understanding of Recovery After Trauma (AURORA) study is a longitudinal study that examined adverse posttraumatic neuropsychiatric sequalae among patients who presented to 28 US urban EDs in the immediate aftermath of a traumatic experience. Enrollment began on September 25, 2017. The 1003 patients considered in this diagnostic/prognostic report completed 3-month assessments by January 31, 2020. Each patient received a baseline ED assessment along with follow-up self-report surveys 2 weeks, 8 weeks, and 3 months later. An ensemble machine learning method was used to predict 3-month PTSD or MDE from baseline information. Data analysis was performed from November 1, 2020, to May 31, 2021. Main Outcomes and Measures: The PTSD Checklist for DSM-5 was used to assess PTSD and the Patient Reported Outcomes Measurement Information System Depression Short-Form 8b to assess MDE. Results: A total of 1003 patients (median [interquartile range] age, 34.5 [24-43] years; 715 [weighted 67.9%] female; 100 [weighted 10.7%] Hispanic, 537 [weighted 52.7%] non-Hispanic Black, 324 [weighted 32.2%] non-Hispanic White, and 42 [weighted 4.4%] of non-Hispanic other race or ethnicity were included in this study. A total of 274 patients (weighted 26.6%) met criteria for 3-month PTSD or MDE. An ensemble machine learning model restricted to 30 predictors estimated in a training sample (patients from the Northeast or Midwest) had good prediction accuracy (mean [SE] area under the curve [AUC], 0.815 [0.031]) and calibration (mean [SE] integrated calibration index, 0.040 [0.002]; mean [SE] expected calibration error, 0.039 [0.002]) in an independent test sample (patients from the South). Patients in the top 30% of predicted risk accounted for 65% of all 3-month PTSD or MDE, with a mean (SE) positive predictive value of 58.2% (6.4%) among these patients at high risk. The model had good consistency across regions of the country in terms of both AUC (mean [SE], 0.789 [0.025] using the Northeast as the test sample and 0.809 [0.023] using the Midwest as the test sample) and calibration (mean [SE] integrated calibration index, 0.048 [0.003] using the Northeast as the test sample and 0.024 [0.001] using the Midwest as the test sample; mean [SE] expected calibration error, 0.034 [0.003] using the Northeast as the test sample and 0.025 [0.001] using the Midwest as the test sample). The most important predictors in terms of Shapley Additive Explanations values were symptoms of anxiety sensitivity and depressive disposition, psychological distress in the 30 days before motor vehicle collision, and peritraumatic psychosomatic symptoms. Conclusions and Relevance: The results of this study suggest that a short set of questions feasible to administer in an ED can predict 3-month PTSD or MDE with good AUC, calibration, and geographic consistency. Patients at high risk can be identified in the ED for targeting if cost-effective preventive interventions are developed.

Optimal individualized decision rules using instrumental variable methods.

There is an extensive literature on the estimation and evaluation of optimal individualized treatment rules in settings where all confounders of the effect of treatment on outcome are observed. We study the development of individualized decision rules in settings where some of these confounders may not have been measured but a valid binary instrument is available for a binary treatment. We first consider individualized treatment rules, which will naturally be most interesting in settings where it is feasible to intervene directly on treatment. We then consider a setting where intervening on treatment is infeasible, but intervening to encourage treatment is feasible. In both of these settings, we also handle the case that the treatment is a limited resource so that optimal interventions focus the available resources on those individuals who will benefit most from treatment. Given a reference rule, we evaluate an optimal individualized rule by its average causal effect relative to a prespecified reference rule. We develop methods to estimate optimal individualized rules and construct asymptotically efficient plug-in estimators of the corresponding average causal effect relative to a prespecified reference rule.

An individualized treatment rule to optimize probability of remission by continuation, switching, or combining antidepressant medications after failing a first-line antidepressant in a two-stage randomized trial.

BACKGROUND: There is growing interest in using composite individualized treatment rules (ITRs) to guide depression treatment selection, but best approaches for doing this are not widely known. We develop an ITR for depression remission based on secondary analysis of a recently published trial for second-line antidepression medication selection using a cutting-edge ensemble machine learning method. METHODS: Data come from the SUN(^_^)D trial, an open-label, assessor blinded pragmatic trial of previously-untreated patients with major depressive disorder from 48 clinics in Japan. Initial clinic-level randomization assigned patients to 50 or 100 mg/day sertraline. We focus on the 1549 patients who failed to remit within 3 weeks and were then rerandomized at the individual-level to continuation with sertraline, switching to mirtazapine, or combining mirtazapine with sertraline. The outcome was remission 9 weeks post-baseline. Predictors included socio-demographics, clinical characteristics, baseline symptoms, changes in symptoms between baseline and week 3, and week 3 side effects. RESULTS: Optimized treatment was associated with significantly increased cross-validated week 9 remission rates in both samples [5.3% (2.4%), p = 0.016 50 mg/day sample; 5.1% (2.7%), p = 0.031 100 mg/day sample] compared to randomization (30.1-30.8%). Optimization was also associated with significantly increased remission in both samples compared to continuation [24.7% in both: 11.2% (3.8%), p = 0.002 50 mg/day sample; 11.7% (3.9%), p = 0.001 100 mg/day sample]. Non-significant gains were found for optimization compared to switching or combining. CONCLUSIONS: An ITR can be developed to improve second-line antidepressant selection, but replication in a larger study with more comprehensive baseline predictors might produce stronger and more stable results.

Correction to: "Optimal Individualized Decision Rules Using Instrumental Variable Methods".

[This corrects the article PMC7959164.].

Prior sleep problems and adverse post-traumatic neuropsychiatric sequelae of motor vehicle collision in the AURORA study.

STUDY OBJECTIVES: Many patients in Emergency Departments (EDs) after motor vehicle collisions (MVCs) develop post-traumatic stress disorder (PTSD) or major depressive episode (MDE). This report from the AURORA study focuses on associations of pre-MVC sleep problems with these outcomes 8 weeks after MVC mediated through peritraumatic distress and dissociation and 2-week outcomes. METHODS: A total of 666 AURORA patients completed self-report assessments in the ED and at 2 and 8 weeks after MVC. Peritraumatic distress, peritraumatic dissociation, and pre-MVC sleep characteristics (insomnia, nightmares, daytime sleepiness, and sleep duration in the 30 days before the MVC, trait sleep stress reactivity) were assessed retrospectively in the ED. The survey assessed acute stress disorder (ASD) and MDE at 2 weeks and at 8 weeks assessed PTSD and MDE (past 30 days). Control variables included demographics, MVC characteristics, and retrospective reports about PTSD and MDE in the 30 days before the MVC. RESULTS: Prevalence estimates were 41.0% for 2-week ASD, 42.0% for 8-week PTSD, 30.5% for 2-week MDE, and 27.2% for 8-week MDE. Pre-MVC nightmares and sleep stress reactivity predicted 8-week PTSD (mediated through 2-week ASD) and MDE (mediated through the transition between 2-week and 8-week MDE). Pre-MVC insomnia predicted 8-week PTSD (mediated through 2-week ASD). Estimates of population attributable risk suggest that blocking effects of sleep disturbance might reduce prevalence of 8-week PTSD and MDE by as much as one-third. CONCLUSIONS: Targeting disturbed sleep in the immediate aftermath of MVC might be one effective way of reducing MVC-related PTSD and MDE.