Improving risk prediction for target subpopulations: Predicting suicidal behaviors among multiple sclerosis patients.

Several recent studies have applied machine learning techniques to develop risk algorithms that predict subsequent suicidal behavior based on electronic health record data. In this study we used a retrospective cohort study design to test whether developing more tailored predictive models-within specific subpopulations of patients-would improve predictive accuracy. A retrospective cohort of 15,117 patients diagnosed with multiple sclerosis (MS), a diagnosis associated with increased risk of suicidal behavior, was used. The cohort was randomly divided into equal sized training and validation sets. Overall, suicidal behavior was identified among 191 (1.3%) of the patients with MS. A Naïve Bayes Classifier model was trained on the training set to predict future suicidal behavior. With 90% specificity, the model detected 37% of subjects who later demonstrated suicidal behavior, on average 4.6 years before the first suicide attempt. The performance of a model trained only on MS patients was better at predicting suicide in MS patients than that a model trained on a general patient sample of a similar size (AUC of 0.77 vs. 0.66). Unique risk factors for suicidal behavior among patients with MS included pain-related codes, gastroenteritis and colitis, and history of smoking. Future studies are needed to further test the value of developing population-specific risk models.

Predicting Suicidal Behavior From Longitudinal Electronic Health Records.

OBJECTIVE: The purpose of this article was to determine whether longitudinal historical data, commonly available in electronic health record (EHR) systems, can be used to predict patients' future risk of suicidal behavior. METHOD: Bayesian models were developed using a retrospective cohort approach. EHR data from a large health care database spanning 15 years (1998-2012) of inpatient and outpatient visits were used to predict future documented suicidal behavior (i.e., suicide attempt or death). Patients with three or more visits (N=1,728,549) were included. ICD-9-based case definition for suicidal behavior was derived by expert clinician consensus review of 2,700 narrative EHR notes (from 520 patients), supplemented by state death certificates. Model performance was evaluated retrospectively using an independent testing set. RESULTS: Among the study population, 1.2% (N=20,246) met the case definition for suicidal behavior. The model achieved sensitive (33%-45% sensitivity), specific (90%-95% specificity), and early (3-4 years in advance on average) prediction of patients' future suicidal behavior. The strongest predictors identified by the model included both well-known (e.g., substance abuse and psychiatric disorders) and less conventional (e.g., certain injuries and chronic conditions) risk factors, indicating that a data-driven approach can yield more comprehensive risk profiles. CONCLUSIONS: Longitudinal EHR data, commonly available in clinical settings, can be useful for predicting future risk of suicidal behavior. This modeling approach could serve as an early warning system to help clinicians identify high-risk patients for further screening. By analyzing the full phenotypic breadth of the EHR, computerized risk screening approaches may enhance prediction beyond what is feasible for individual clinicians.