Improving Methods of Identifying Anaphylaxis for Medical Product Safety Surveillance Using Natural Language Processing and Machine Learning.

We sought to determine whether machine learning and natural language processing (NLP) applied to electronic medical records could improve performance of automated health-care claims-based algorithms to identify anaphylaxis events using data on 516 patients with outpatient, emergency department, or inpatient anaphylaxis diagnosis codes during 2015-2019 in 2 integrated health-care institutions in the Northwest United States. We used one site's manually reviewed gold-standard outcomes data for model development and the other's for external validation based on cross-validated area under the receiver operating characteristic curve (AUC), positive predictive value (PPV), and sensitivity. In the development site 154 (64%) of 239 potential events met adjudication criteria for anaphylaxis compared with 180 (65%) of 277 in the validation site. Logistic regression models using only structured claims data achieved a cross-validated AUC of 0.58 (95% CI: 0.54, 0.63). Machine learning improved cross-validated AUC to 0.62 (0.58, 0.66); incorporating NLP-derived covariates further increased cross-validated AUCs to 0.70 (0.66, 0.75) in development and 0.67 (0.63, 0.71) in external validation data. A classification threshold with cross-validated PPV of 79% and cross-validated sensitivity of 66% in development data had cross-validated PPV of 78% and cross-validated sensitivity of 56% in external data. Machine learning and NLP-derived data improved identification of validated anaphylaxis events.

Medical chart validation of inpatient diagnosis codes for transfusion-related acute lung injury 2013-2015.

INTRODUCTION: Transfusion-related acute lung injury (TRALI), an adverse event occurring during or within 6 hours of transfusion, is a leading cause of transfusion-associated fatalities reported to the US Food and Drug Administration. There is limited information on the validity of diagnosis codes for TRALI recorded in inpatient electronic medical records (EMRs). STUDY DESIGNS AND METHODS: We conducted a validation study to establish the positive predictive value (PPV) of TRALI International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes recorded within a large hospital system between 2013 and 2015. A physician with critical care expertise confirmed the TRALI diagnosis. As TRALI is likely underdiagnosed, we used the specific code (518.7), and codes for respiratory failure (518.82) in combination with transfusion reaction (999.80, 999.89, E934.7). RESULTS: Among almost four million inpatient stays, we identified 208 potential TRALI cases with ICD-9-CM codes and reviewed 195 medical records; 68 (35%) met clinical definitions for TRALI (26 [38%] definitive, 15 [22%] possible, 27 [40%] delayed). Overall, the PPV for all inpatient TRALI diagnoses was 35% (95% confidence interval (CI), 28-42). The PPV for the TRALI-specific code was 44% (95% CI, 35-54). CONCLUSION: We observed low PPVs (<50%) for TRALI ICD-9-CM diagnosis codes as validated by medical charts, which may relate to inconsistent code use, incomplete medical records, or other factors. Future studies using TRALI diagnosis codes in EMR databases may consider confirming diagnoses with medical records, assessing TRALI ICD, Tenth Revision, Clinical Modification codes, or exploring alternative ways for of accurately identifying TRALI in EMR databases. KEY POINTS: In 169 hospitals, we identified 208 potential TRALI cases, reviewed 195 charts, and confirmed 68 (35%) cases met TRALI clinical definitions. As many potential TRALI cases identified with diagnosis codes did not meet clinical definitions, medical record confirmation may be prudent.

Validity of ICD-10-CM diagnoses to identify hospitalizations for serious infections among patients treated with biologic therapies.

PURPOSE: Identifying hospitalizations for serious infections among patients dispensed biologic therapies within healthcare databases is important for post-marketing surveillance of these drugs. We determined the positive predictive value (PPV) of an ICD-10-CM-based diagnostic coding algorithm to identify hospitalization for serious infection among patients dispensed biologic therapy within the FDA's Sentinel Distributed Database. METHODS: We identified health plan members who met the following algorithm criteria: (1) hospital ICD-10-CM discharge diagnosis of serious infection between July 1, 2016 and August 31, 2018; (2) either outpatient/emergency department infection diagnosis or outpatient antimicrobial treatment within 7 days prior to hospitalization; (3) inflammatory bowel disease, psoriasis, or rheumatological diagnosis within 1 year prior to hospitalization, and (4) were dispensed outpatient biologic therapy within 90 days prior to admission. Medical records were reviewed by infectious disease clinicians to adjudicate hospitalizations for serious infection. The PPV (95% confidence interval [CI]) for confirmed events was determined after further weighting by the prevalence of the type of serious infection in the database. RESULTS: Among 223 selected health plan members who met the algorithm, 209 (93.7% [95% CI, 90.1%-96.9%]) were confirmed to have a hospitalization for serious infection. After weighting by the prevalence of the type of serious infection, the PPV of the ICD-10-CM algorithm identifying a hospitalization for serious infection was 80.2% (95% CI, 75.3%-84.7%). CONCLUSIONS: The ICD-10-CM-based algorithm for hospitalization for serious infection among patients dispensed biologic therapies within the Sentinel Distributed Database had 80% PPV for confirmed events and could be considered for use within pharmacoepidemiologic studies.

Extension of Disease Risk Score-Based Confounding Adjustments for Multiple Outcomes of Interest: An Empirical Evaluation.

Use of disease risk score (DRS)-based confounding adjustment when estimating treatment effects on multiple outcomes is not well studied. We designed an empirical cohort study to compare dabigatran initiators and warfarin initiators with respect to risks of ischemic stroke and major bleeding in 12 sequential monitoring periods (90 days each), using data from the Truven Marketscan database (Truven Health Analytics, Ann Arbor, Michigan). We implemented 2 approaches to combine DRS for multiple outcomes: 1) 1:1 matching on prognostic propensity scores (PPS), created using DRS for bleeding and stroke as independent variables in a propensity score (PS) model; and 2) simultaneous 1:1 matching on DRS for bleeding and stroke using Mahalanobis distance (M-distance), and compared their performance with that of traditional PS matching. M-distance matching appeared to produce more stable results in the early marketing period than both PPS and traditional PS matching; hazard ratios from unadjusted analysis, traditional PS matching, PPS matching, and M-distance matching after 4 periods were 0.72 (95% confidence interval (CI): 0.51, 1.03), 0.61 (95% CI: 0.31, 1.09), 0.55 (95% CI: 0.33, 0.91), and 0.78 (95% CI: 0.45, 1.34), respectively, for stroke and 0.65 (95% CI: 0.53, 0.80), 0.78 (95% CI: 0.60, 1.01), 0.75 (95% CI: 0.59, 0.96), and 0.78 (95% CI: 0.64, 0.95), respectively, for bleeding. In later periods, estimates were similar for traditional PS matching and M-distance matching but suggested potential residual confounding with PPS matching. These results suggest that M-distance matching may be a valid approach for extension of DRS-based confounding adjustments for multiple outcomes of interest.

Progression of Heart Failure in People with Type 2 Diabetes in Germany: An Analysis Using German Health Insurance Claims Data.

Background: Individuals with type 2 diabetes (T2D) show high risk of heart failure (HF). Left ventricular ejection fraction is a major factor for disease progression. In Germany, no recent longitudinal data are available. Objectives: To (1) measure the proportion of individuals with T2D who acquire HF over 2 years and (2) categorize ejection fraction using routine data and an algorithm, and (3) understand progression of HF in 5-year follow-up. Methods: This descriptive, retrospective study used longitudinal data from German statutory health insurance claims. A model using coded data classified the patients with HF into ejection fraction (EF) categories. Individuals were selected during 2013, with an inclusion period from 2014 to 2015 and a follow-up from 2016 to 2020. Baseline characteristics included demographic data, disease stage, comorbidities, and risk factors. Follow-up criteria included major adverse cardiac events (MACEs), EF category, and mortality. Disease progression was visualized by Sankey plots. Results: Among the 173 195 individuals with T2D identified in 2013, 6725 (median age, 74 years) developed HF in 2014 or 2015. 34.4% of individuals had MACEs, and 42.9% died over 5 years. Myocardial infarction (42%) was the most common event, followed by stroke (32%) and hospitalization (28%). A total of 5282 (78.54%) patients were classified into preserved EF and 1443 (21.46%) into reduced EF. Survival after 5 years was 71% in HF for preserved EF patients, and 29% in the HF for those with reduced EF. Conclusion: Heart failure is relevant in individuals with diabetes. A high number of patients may likely not survive a 5-year period. Validation of the model with German data is highly desirable. New ways of close monitoring could help improve outcomes.

A Systematic Literature Review of Psychological Interventions for Adults with Type 1 Diabetes.

INTRODUCTION: The psychological burden of type 1 diabetes mellitus (T1DM) is considerable. The condition affects the daily lives of adults living with T1DM (ALWT1DM) in many ways. International guidelines highlight the importance of providing psychological support to ALWT1DM to improve health outcomes and well-being. METHODS: We conducted a systematic literature review of randomised controlled trials (RCTs) to identify the evidence on the impact of psychological interventions on glycaemic control and psychological outcomes in ALWT1DM. Literature searches of Medline, Embase, Cochrane Central Register of Controlled Trials, PsycInfo, and the grey literature were performed to identify relevant RCTs, published in English, from 2001 onward. Fourteen RCTs of ten psychological interventions in ALWT1DM were eligible and included in the qualitative synthesis. The studies varied considerably in terms of duration, target population, endpoints, and efficacy. RESULTS: Overall, psychological interventions did not perform significantly better than control treatments in improving glycaemic control, although selected patient groups did report benefits from some psychological intervention types, such as cognitive behavioural therapy. Although most of the psychological interventions produced small, nonsignificant improvements in self-reported patient functioning, some treatments were effective in reducing diabetes distress and improving mental health, even if no impact on glycaemic control was observed. DISCUSSION: Current guidelines for the treatment of T1DM recommend access to psychological services; however, there is a paucity of high-quality evidence from clinical trials on the effectiveness or preferred structure of psychological support. There is a clear need for more rigorous, large-scale, international research to address the efficacy of psychological interventions in ALWT1DM.

Behavioral Engagement and Activation Model Study (BEAMS): A latent class analysis of adopters and non-adopters of digital health technologies among people with Type 2 diabetes.

Many people with Type 2 diabetes (T2D) who could benefit from digital health technologies (DHTs) are either not using DHTs or do use them, but not for long enough to reach their behavioral or metabolic goals. We aimed to identify subgroups within DHT adopters and non-adopters and describe their unique profiles to better understand the type of tailored support needed to promote effective and sustained DHT use across a diverse T2D population. We conducted latent class analysis of a sample of adults with T2D who responded to an internet survey between December 2021 and March 2022. We describe the clinical and psychological characteristics of DHT adopters and non-adopters, and their attitudes toward DHTs. A total of 633 individuals were characterized as either DHT "Adopters" (n = 376 reporting any use of DHT) or "Non-Adopters" (n = 257 reporting never using any DHT). Within Adopters, three subgroups were identified: 21% (79/376) were "Self-managing Adopters," who reported high health activation and self-efficacy for diabetes management, 42% (158/376) were "Activated Adopters with dropout risk," and 37% (139/376) were "Non-Activated Adopters with dropout risk." The latter two subgroups reported barriers to using DHTs and lower rates of intended future use. Within Non-Adopters, two subgroups were identified: 31% (79/257) were "Activated Non-Adopters," and 69% (178/257) were "Non-Adopters with barriers," and were similarly distinguished by health activation and barriers to using DHTs. Beyond demographic characteristics, psychological, and clinical factors may help identify different subgroups of Adopters and Non-Adopters.

In this study, we characterized subgroups of adopters and non-adopters of digital health technologies (DHTs) for managing Type 2 diabetes, such as apps to track nutrition, continuous glucose monitors, and activity monitors like Fitbit. Self-efficacy for diabetes management, health activation, and perceived barriers to use DHT emerged as characteristics that distinguished subgroups. Notably, subgroups of adopters differed in their interest to use these technologies in the next 3 months; groups with low levels of self-efficacy and health activation were least interested in using them and thus at risk of discontinuing use. The ability to identify these subgroups can inform strategies tailored to each subgroup that motivate adoption of DHTs and promote long-term engagement.

Patient and Physician Perspectives on the Use of a Connected Ecosystem for Diabetes Management: International Cross-Sectional Observational Study.

BACKGROUND: Collaboration between people with type 2 diabetes (T2DM) and their health care teams is important for optimal control of the disease and outcomes. Digital technologies could potentially tie together several health care-related devices and platforms into connected ecosystems (CES), but attitudes about CES are unknown. OBJECTIVE: We surveyed convenience samples of patients and physicians to better understand which patient characteristics are associated with higher likelihoods of (1) participating in a potential CES program, as self-reported by patients with T2DM and (2) clinical benefit from participation in a potential CES program, as reported by physicians. METHODS: Adults self-reporting a diagnosis of T2DM and current insulin use (n=197), and 33 physicians whose practices included ≥20% of such patients, were enrolled in the United States, France, and Germany. We surveyed both groups about the likelihood of patient participation in a CES. We then examined the associations between patients' clinical and sociodemographic characteristics and this likelihood. We also described characteristics of patients likely to clinically benefit from CES use, according to physicians. RESULTS: Compared with patients in Germany and France, US patients were younger (mean age 45.3 [SD 11.9] years vs 61.9 [SD 9.2] and 65.8 [SD 9.4] years, respectively), more often female, more highly educated, and more often working full-time. In all, 51 (44.7%) US patients, 16 (36.4%) German patients, and 18 (46.3%) French patients indicated strong interest in a CES program, and 115 (78.7%) reported currently using ≥1 connected device or app. However, physicians believed that only 11.3%-19.2% of their patients were using connected devices or apps to manage their disease. Physicians also reported infrequently recommending or prescribing connected devices to their patients, although ≥80% (n=28) of them thought that a CES could help support their patients in managing their disease. The factors most predictive of patient likelihood of participating in a CES program were cost, inclusion of medication reminders, and linking blood glucose levels to behaviors such as eating and exercise. In all countries, the most common patient expectations for a CES program were that it could help them eat more healthfully, increase their physical activity, increase their understanding of how blood glucose relates to behavior such as exercise and eating, and reduce stress. Physicians thought that newly diagnosed patients, sicker patients-those who had been hospitalized for diabetes, were currently using insulin, or who had any comorbid condition-and patients who were nonadherent to treatment were most likely to benefit from CES use. CONCLUSIONS: In this study, there was a high degree of interest in the future use of CES, although additional education is needed among both patients with T2DM and their physicians to achieve the full potential of such systems to improve self-management and clinical care for the disease.

Electronic phenotyping of health outcomes of interest using a linked claims-electronic health record database: Findings from a machine learning pilot project.

OBJECTIVE: Claims-based algorithms are used in the Food and Drug Administration Sentinel Active Risk Identification and Analysis System to identify occurrences of health outcomes of interest (HOIs) for medical product safety assessment. This project aimed to apply machine learning classification techniques to demonstrate the feasibility of developing a claims-based algorithm to predict an HOI in structured electronic health record (EHR) data. MATERIALS AND METHODS: We used the 2015-2019 IBM MarketScan Explorys Claims-EMR Data Set, linking administrative claims and EHR data at the patient level. We focused on a single HOI, rhabdomyolysis, defined by EHR laboratory test results. Using claims-based predictors, we applied machine learning techniques to predict the HOI: logistic regression, LASSO (least absolute shrinkage and selection operator), random forests, support vector machines, artificial neural nets, and an ensemble method (Super Learner). RESULTS: The study cohort included 32 956 patients and 39 499 encounters. Model performance (positive predictive value [PPV], sensitivity, specificity, area under the receiver-operating characteristic curve) varied considerably across techniques. The area under the receiver-operating characteristic curve exceeded 0.80 in most model variations. DISCUSSION: For the main Food and Drug Administration use case of assessing risk of rhabdomyolysis after drug use, a model with a high PPV is typically preferred. The Super Learner ensemble model without adjustment for class imbalance achieved a PPV of 75.6%, substantially better than a previously used human expert-developed model (PPV = 44.0%). CONCLUSIONS: It is feasible to use machine learning methods to predict an EHR-derived HOI with claims-based predictors. Modeling strategies can be adapted for intended uses, including surveillance, identification of cases for chart review, and outcomes research.

Evaluation of Use of Technologies to Facilitate Medical Chart Review.

INTRODUCTION: While medical chart review remains the gold standard to validate health conditions or events identified in administrative claims and electronic health record databases, it is time consuming, expensive and can involve subjective decisions. AIM: The aim of this study was to describe the landscape of technology-enhanced approaches that could be used to facilitate medical chart review within and across distributed data networks. METHOD: We conducted a semi-structured survey regarding processes for medical chart review with organizations that either routinely do medical chart review or use technologies that could facilitate chart review. RESULTS: Fifteen out of 17 interviewed organizations used optical character recognition (OCR) or natural language processing (NLP) in their chart review process. None used handwriting recognition software. While these organizations found OCR and NLP to be useful for expediting extraction of useful information from medical charts, they also mentioned several challenges. Quality of medical scans can be variable, interfering with the accuracy of OCR. Additionally, linguistic complexity in medical notes and heterogeneity in reporting templates used by different healthcare systems can reduce the transportability of NLP-based algorithms to diverse healthcare settings. CONCLUSION: New technologies including OCR and NLP are currently in use by various organizations involved in medical chart review. While technology-enhanced approaches could scale up capacity to validate key variables and make information about important clinical variables from medical records more generally available for research purposes, they often require considerable customization when employed in a distributed data environment with multiple, diverse healthcare settings.