Validation of the Enhanced Opioid Identification and Co-occurring Disorders Algorithms.

Objectives This report documents the results of a validation study conducted to assess the reliability of two algorithms applied to the 2016 National Hospital Care Survey. One algorithm identifies opioid-involved and opioid overdose hospital encounters, and the other identifies encounters with patients that have substance use disorders and selected mental health issues. These algorithms use both medical codes and natural language processing to identify encounters. Methods To validate the algorithms, medical record abstraction was performed on a stratified sample of 900 hospital encounters from the 2016 National Hospital Care Survey. The abstractors recorded their determinations of opioid involvement, opioid overdose, substance use disorder, and mental health issues on a standard form. Abstractors' determinations were compared with algorithm output to assess the overall performance using F-score and Matthews correlation coefficient. The latter provided a secondary measure of performance. The 2016 National Hospital Care Survey data are unweighted and not nationally representative. Results Overall algorithm performance varied by topic and by metric. The opioid-involvement algorithm achieved the highest performance, performing well with an F-score of 0.95, followed by the substance use disorder algorithm (F-score of 0.79), the mental health issues algorithm (F-score of 0.68), and the opioid overdose algorithm (F-score of 0.48). Assessment by Matthews correlation coefficient indicated an overall poorer level of performance, ranging from a high of 0.57 for the mental health issues algorithm to a low of 0.33 for the opioid-involvement algorithm. The causes of false positives and false negatives likewise varied, including both overly broad code and keyword inclusions as well as incompleteness of data submitted to the National Hospital Care Survey. Conclusion The validation study illustrates which aspects of the developed algorithms performed well and which aspects should be altered or discarded in future iterations. It further emphasizes the importance of data completeness, therefore laying the groundwork for improvements to future survey analyses.

Identifying Co-occurring Disorders Among Patients With an Opioid involved Hospital Encounter Using National Hospital Care Survey Data.

This report documents the development of the 2016 National Hospital Care Survey (NHCS) Co-occurring Disorders Algorithm, which can be used to identify patients with an opioid-involved hospital encounter who had lifetime diagnoses of both a substance use disorder and a selected mental health issue. Lifetime diagnoses are defined as diagnoses at any point in the past or during the current encounter. This algorithm was created to complement the earlier NHCS Enhanced Opioid Identification Algorithm designed to improve the classification of patients with opioid-involved hospital encounters.

Enhancing Identification of Opioid-involved Health Outcomes Using National Hospital Care Survey Data.

Objectives This report documents the development of the 2016 National Hospital Care Survey (NHCS) Enhanced Opioid Identification Algorithm, an algorithm that can be used to identify opioid-involved and opioid overdose hospital encounters. Additionally, the algorithm can be used to identify opioids and opioid antagonists that can be used to reverse opioid overdose (naloxone) and to treat opioid use disorder (naltrexone).