Using Administrative Data to Predict Suicide After Psychiatric Hospitalization in the Veterans Health Administration System.

There is a very high suicide rate in the year after psychiatric hospital discharge. Intensive postdischarge case management programs can address this problem but are not cost-effective for all patients. This issue can be addressed by developing a risk model to predict which inpatients might need such a program. We developed such a model for the 391,018 short-term psychiatric hospital admissions of US veterans in Veterans Health Administration (VHA) hospitals 2010-2013. Records were linked with the National Death Index to determine suicide within 12 months of hospital discharge (n=771). The Super Learner ensemble machine learning method was used to predict these suicides for time horizon between 1 week and 12 months after discharge in a 70% training sample. Accuracy was validated in the remaining 30% holdout sample. Predictors included VHA administrative variables and small area geocode data linked to patient home addresses. The models had AUC=.79-.82 for time horizons between 1 week and 6 months and AUC=.74 for 12 months. An analysis of operating characteristics showed that 22.4%-32.2% of patients who died by suicide would have been reached if intensive case management was provided to the 5% of patients with highest predicted suicide risk. Positive predictive value (PPV) at this higher threshold ranged from 1.2% over 12 months to 3.8% per case manager year over 1 week. Focusing on the low end of the risk spectrum, the 40% of patients classified as having lowest risk account for 0%-9.7% of suicides across time horizons. Variable importance analysis shows that 51.1% of model performance is due to psychopathological risk factors accounted, 26.2% to social determinants of health, 14.8% to prior history of suicidal behaviors, and 6.6% to physical disorders. The paper closes with a discussion of next steps in refining the model and prospects for developing a parallel precision treatment model.

Development and Validation of a Machine Learning Individualized Treatment Rule in First-Episode Schizophrenia.

Importance: Little guidance exists to date on how to select antipsychotic medications for patients with first-episode schizophrenia. Objective: To develop a preliminary individualized treatment rule (ITR) for patients with first-episode schizophrenia. Design, Setting, and Participants: This prognostic study obtained data from Taiwan's National Health Insurance Research Database on patients with prescribed antipsychotic medications, ambulatory claims, or discharge diagnoses of a schizophrenic disorder between January 1, 2005, and December 31, 2011. An ITR was developed by applying a targeted minimum loss-based ensemble machine learning method to predict treatment success from baseline clinical and demographic data in a 70% training sample. The model was validated in the remaining 30% of the sample. The probability of treatment success was estimated for each medication for each patient under the model. The analysis was conducted between July 16, 2018, and July 15, 2019. Exposures: Fifteen different antipsychotic medications. Main Outcomes and Measures: Treatment success was defined as not switching medication and not being hospitalized for 12 months. Results: Among the 32 277 patients in the analysis, the mean (SD) age was 36.7 (14.3) years, and 15 752 (48.8%) were male. In the validation sample, the treatment success rate (SE) was 51.7% (1.0%) under the ITR and was 44.5% (0.5%) in the observed population (Z = 7.1; P < .001). The estimated treatment success if all patients were given a prescription for 1 medication was significantly lower for each of the 13 medications than under the ITR (Z = 4.2-16.8; all P < .001). Aripiprazole (3088 [31.9%]) and amisulpride (2920 [30.2%]) were the medications most often recommended by the ITR. Only 1054 patients (10.9%) received ITR-recommended medications. Observed treatment success, although lower than the success under the ITR, was nonetheless significantly higher than if medications had been randomized (44.5% [SE, 0.55%] vs 41.3% [SE, 0.4%]; Z = 6.9; P < .001), although only marginally higher than if medications had been randomized in their observed population proportions (44.5% [SE, 0.5%] vs 43.5% [SE, 0.4%]; Z = 2.2; P = .03]). Conclusions and Relevance: These results suggest that an ITR may be associatded with an increase in the treatment success rate among patients with first-episode schizophrenia, but experimental evaluation is needed to confirm this possibility. If confirmed, model refinement that investigates biomarkers, clinical observations, and patient reports as additional predictors in iterative pragmatic trials would be needed before clinical implementation.

A Preliminary Precision Treatment Rule for Remission of Suicide Ideation.

OBJECTIVE: There is growing interest in the development of composite precision treatment rules (PTRs) to guide the selection of the treatments most likely to be helpful for individual patients. We present here the results of an effort to develop a preliminary PTR for Collaborative Assessment and Management of Suicidality (CAMS) relative to enhanced-care as usual based on secondary analysis of the Operation Worth Living (OWL) randomized controlled trial. The outcome of interest is eliminating suicide ideation (SI) within 3 months of initiating treatment. METHOD: A state-of-the-art ensemble machine learning method was used to develop the PTR among the n = 148 U.S. Soldiers (predominately male and White, age range 18-48) OWL patients. RESULTS: We estimated that CAMS was the better treatment for 77.8% of patients and that treatment assignment according to the PTR would result in a 13.6% (95% CI: 0.9%-26.3%) increase in 3-month SI remission compared to random treatment assignment. CONCLUSIONS: Although promising, results are limited by the small sample size, restrictive baseline assessment, and inability to evaluate effects on suicidal behaviors or disaggregate based on history of suicidal behaviors. Replication is needed in larger samples with comprehensive baseline assessments, longer-term follow-ups, and more extensive outcomes.

Selecting Optimal Subgroups for Treatment Using Many Covariates.

We consider the problem of selecting the optimal subgroup to treat when data on covariates are available from a randomized trial or observational study. We distinguish between four different settings including: (1) treatment selection when resources are constrained; (2) treatment selection when resources are not constrained; (3) treatment selection in the presence of side effects and costs; and (4) treatment selection to maximize effect heterogeneity. We show that, in each of these cases, the optimal treatment selection rule involves treating those for whom the predicted mean difference in outcomes comparing those with versus without treatment, conditional on covariates, exceeds a certain threshold. The threshold varies across these four scenarios, but the form of the optimal treatment selection rule does not. The results suggest a move away from the traditional subgroup analysis for personalized medicine. New randomized trial designs are proposed so as to implement and make use of optimal treatment selection rules in healthcare practice.

Targeted Estimation of the Relationship Between Childhood Adversity and Fluid Intelligence in a US Population Sample of Adolescents.

Many studies have shown inverse associations between childhood adversity and intelligence, although most are based on small clinical samples and fail to account for the effects of multiple co-occurring adversities. Using data from the 2001-2004 National Comorbidity Survey Adolescent Supplement, a cross-sectional US population study of adolescents aged 13-18 years (n = 10,073), we examined the associations between 11 childhood adversities and intelligence, using targeted maximum likelihood estimation. Targeted maximum likelihood estimation incorporates machine learning to identify the relationships between exposures and outcomes without overfitting, including interactions and nonlinearity. The nonverbal score from the Kaufman Brief Intelligence Test was used as a standardized measure of fluid reasoning. Childhood adversities were grouped into deprivation and threat types based on recent conceptual models. Adjusted marginal mean differences compared the mean intelligence score if all adolescents experienced each adversity to the mean in the absence of the adversity. The largest associations were observed for deprivation-type experiences, including poverty and low parental education, which were related to reduced intelligence. Although lower in magnitude, threat events related to intelligence included physical abuse and witnessing domestic violence. Violence prevention and poverty-reduction measures would likely improve childhood cognitive outcomes.