Machine Learning Prediction of Suicide Risk Does Not Identify Patients Without Traditional Risk Factors.

Objective: To determine whether predictions of suicide risk from machine learning models identify unexpected patients or patients without medical record documentation of traditional risk factors.Methods: The study sample included 27,091,382 outpatient mental health (MH) specialty or general medical visits with a MH diagnosis for patients aged 11 years or older from January 1, 2009, to September 30, 2017. We used predicted risk scores of suicide attempt and suicide death, separately, within 90 days of visits to classify visits into risk score percentile strata. For each stratum, we calculated counts and percentages of visits with traditional risk factors, including prior self-harm diagnoses and emergency department visits or hospitalizations with MH diagnoses, in the last 3, 12, and 60 months.Results: Risk-factor percentages increased with predicted risk scores. Among MH specialty visits, 66%, 88%, and 99% of visits with suicide attempt risk scores in the top 3 strata (respectively, 90th-95th, 95th-98th, and ≥ 98th percentiles) and 60%, 77%, and 93% of visits with suicide risk scores in the top 3 strata represented patients who had at least one traditional risk factor documented in the prior 12 months. Among general medical visits, 52%, 66%, and 90% of visits with suicide attempt risk scores in the top 3 strata and 45%, 66%, and 79% of visits with suicide risk scores in the top 3 strata represented patients who had a history of traditional risk factors in the last 12 months.Conclusions: Suicide risk alerts based on these machine learning models coincide with patients traditionally thought of as high-risk at their high-risk visits.