Improving Methods of Identifying Anaphylaxis for Medical Product Safety Surveillance Using Natural Language Processing and Machine Learning.

We sought to determine whether machine learning and natural language processing (NLP) applied to electronic medical records could improve performance of automated health-care claims-based algorithms to identify anaphylaxis events using data on 516 patients with outpatient, emergency department, or inpatient anaphylaxis diagnosis codes during 2015-2019 in 2 integrated health-care institutions in the Northwest United States. We used one site's manually reviewed gold-standard outcomes data for model development and the other's for external validation based on cross-validated area under the receiver operating characteristic curve (AUC), positive predictive value (PPV), and sensitivity. In the development site 154 (64%) of 239 potential events met adjudication criteria for anaphylaxis compared with 180 (65%) of 277 in the validation site. Logistic regression models using only structured claims data achieved a cross-validated AUC of 0.58 (95% CI: 0.54, 0.63). Machine learning improved cross-validated AUC to 0.62 (0.58, 0.66); incorporating NLP-derived covariates further increased cross-validated AUCs to 0.70 (0.66, 0.75) in development and 0.67 (0.63, 0.71) in external validation data. A classification threshold with cross-validated PPV of 79% and cross-validated sensitivity of 66% in development data had cross-validated PPV of 78% and cross-validated sensitivity of 56% in external data. Machine learning and NLP-derived data improved identification of validated anaphylaxis events.

Relationship between cardiovascular risk and lipid testing in one health care system: a retrospective cohort study.

BACKGROUND: The US Preventive Services Taskforce (USPSTF) recommends routine lipid screening beginning age 35 for men [1]. For women age 20 and older, as well as men age 20-34, screening is recommended if cardiovascular risk factors are present. Prior research has focused on underutilization but not overuse of lipid testing. The objective is to document over- and under-use of lipid testing in an insured population of persons at low, moderate and high cardiovascular disease (CVD) risk for persons not already on statins. METHODS: The study is a retrospective cohort study that included all adults without prior CVD who were continuously enrolled in a large integrated healthcare system from 2005 to 2010. Measures included lipid test frequency extracted from administrative data and Framingham cardiovascular risk equations applied using electronic medical record data. Five year lipid testing patterns were examined by age, sex and CVD risk. Generalized linear models were used to estimate the relative risk for over testing associated with patient characteristics. RESULTS: Among males and females for whom testing is not recommended, 35.8 % and 61.5 % received at least one lipid test in the prior 5 years and 8.4 % and 24.4 % had two or more. Over-testing was associated with age, race, comorbidity, primary care use and neighborhood income. Among individuals at moderate and high-risk (not already treated with statins) and for whom screening is recommended, between 21.4 % and 25.1 % of individuals received no screening in the prior 5 years. CONCLUSIONS: Based on USPSTF lipid screening recommendations, this study documents substantial over-testing among individuals with low CVD risk and under-testing among individuals with moderate to high-risk not already on statins. Opportunity exists to better focus lipid screening efforts appropriate to CVD risk.

Clinician identification of youth abusing over-the-counter products for weight control in a large U.S. integrated health system.

BACKGROUND: Abuse of over-the-counter (OTC) products, such as diet pills and laxatives, for weight control by adolescents is well-documented and can precipitate serious medical conditions. Yet only a small percentage of youth with disordered weight control behaviors receive treatment. The objective of this study was to examine how often clinicians communicate with youth with symptoms consistent with abuse of OTC products for weight control about possible use of these products. We used electronic medical records and administrative claims for services for 53,229 12 to 17 year old patients receiving care from an integrated health system in the U.S. Northwest from August 2007 to December 2010. We examined electronic text of clinical notes to identify encounters in which the clinician noted one of 10 metabolic conditions potentially associated with abuse of OTC products (diet pills, laxatives, diuretics, ipecac, orlistat, and alli®) for weight control and then assessed whether clinicians noted communication with adolescent patients about possible use of OTC products for weight control. RESULTS: We identified 130 (0.2% of sample) patients with clinical notes indicating one or more of the metabolic conditions. In clinical notes for only four (3.1%) of these patients did clinicians document suspicion or communication about possible abuse of the OTC products. All four had a previous eating disorder diagnosis. In the 12 months subsequent to the clinical encounter in which a metabolic disturbance was identified, medical notes for only three (2.3%) of the 130 patients indicated clinician suspicion or communication about possible abuse of these products or an eating disorder. CONCLUSIONS: Clinicians are missing a critical window of opportunity to query adolescents when presenting with suspicious metabolic disturbances about possible abuse of OTC products for weight control. Clinicians may need more training to detect OTC product abuse, and electronic medical records should prompt more thorough enquiry.