Improving Methods of Identifying Anaphylaxis for Medical Product Safety Surveillance Using Natural Language Processing and Machine Learning.

We sought to determine whether machine learning and natural language processing (NLP) applied to electronic medical records could improve performance of automated health-care claims-based algorithms to identify anaphylaxis events using data on 516 patients with outpatient, emergency department, or inpatient anaphylaxis diagnosis codes during 2015-2019 in 2 integrated health-care institutions in the Northwest United States. We used one site's manually reviewed gold-standard outcomes data for model development and the other's for external validation based on cross-validated area under the receiver operating characteristic curve (AUC), positive predictive value (PPV), and sensitivity. In the development site 154 (64%) of 239 potential events met adjudication criteria for anaphylaxis compared with 180 (65%) of 277 in the validation site. Logistic regression models using only structured claims data achieved a cross-validated AUC of 0.58 (95% CI: 0.54, 0.63). Machine learning improved cross-validated AUC to 0.62 (0.58, 0.66); incorporating NLP-derived covariates further increased cross-validated AUCs to 0.70 (0.66, 0.75) in development and 0.67 (0.63, 0.71) in external validation data. A classification threshold with cross-validated PPV of 79% and cross-validated sensitivity of 66% in development data had cross-validated PPV of 78% and cross-validated sensitivity of 56% in external data. Machine learning and NLP-derived data improved identification of validated anaphylaxis events.

Validation of Acute Pancreatitis Among Adults in an Integrated Healthcare System.

BACKGROUND: Acute pancreatitis is a serious gastrointestinal disease that is an important target for drug safety surveillance. Little is known about the accuracy of ICD-10 codes for acute pancreatitis in the United States, or their performance in specific clinical settings. We conducted a validation study to assess the accuracy of acute pancreatitis ICD-10 diagnosis codes in inpatient, emergency department (ED), and outpatient settings. METHODS: We reviewed electronic medical records for encounters with acute pancreatitis diagnosis codes in an integrated healthcare system from October 2015 to December 2019. Trained abstractors and physician adjudicators determined whether events met criteria for acute pancreatitis. RESULTS: Out of 1,844 eligible events, we randomly sampled 300 for review. Across all clinical settings, 182 events met validation criteria for an overall positive predictive value (PPV) of 61% (95% confidence intervals [CI] = 55, 66). The PPV was 87% (95% CI = 79, 92%) for inpatient codes, but only 45% for ED (95% CI = 35, 54%) and outpatient (95% CI = 34, 55%) codes. ED and outpatient encounters accounted for 43% of validated events. Acute pancreatitis codes from any encounter type with lipase >3 times the upper limit of normal had a PPV of 92% (95% CI = 86, 95%) and identified 85% of validated events (95% CI = 79, 89%), while codes with lipase <3 times the upper limit of normal had a PPV of only 22% (95% CI = 16, 30%). CONCLUSIONS: These results suggest that ICD-10 codes accurately identified acute pancreatitis in the inpatient setting, but not in the ED and outpatient settings. Laboratory data substantially improved algorithm performance.

Health Care Utilization in the 6 Months Following SARS-CoV-2 Infection.

Importance: After SARS-CoV-2 infection, many patients present with persistent symptoms for at least 6 months, collectively termed post-COVID conditions (PCC). However, the impact of PCC on health care utilization has not been well described. Objectives: To estimate COVID-19-associated excess health care utilization following acute SARS-CoV-2 infection and describe utilization for select PCCs among patients who had positive SARS-CoV-2 test results (including reverse transcription-polymerase chain reaction and antigen tests) compared with control patients whose results were negative. Design, Setting, and Participants: This matched retrospective cohort study included patients of all ages from 8 large integrated health care systems across the United States who completed a SARS-CoV-2 diagnostic test during March 1 to November 1, 2020. Patients were matched on age, sex, race and ethnicity, site, and date of SARS-CoV-2 test and were followed-up for 6 months. Data were analyzed from March 18, 2021, to June 8, 2022. Exposure: SARS-CoV-2 infection. Main Outcomes and Measures: Ratios of rate ratios (RRRs) for COVID-19-associated health care utilization were calculated with a difference-in-difference analysis using Poisson regression models. RRRs were estimated overall, by health care setting, by select population characteristics, and by 44 PCCs. COVID-19-associated excess health care utilization was estimated by health care setting. Results: The final matched cohort included 127 859 patients with test results positive for SARS-CoV-2 and 127 859 patients with test results negative for SARS-CoV-2. The mean (SD) age of the study population was 41.2 (18.6) years, 68 696 patients in each group (53.7%) were female, and each group included 66 211 Hispanic patients (51.8%), 9122 non-Hispanic Asian patients (7.1%), 7983 non-Hispanic Black patients (6.2%), and 34 326 non-Hispanic White patients (26.9%). Overall, SARS-CoV-2 infection was associated with a 4% increase in health care utilization over 6 months (RRR, 1.04 [95% CI, 1.03-1.05]), predominantly for virtual encounters (RRR, 1.14 [95% CI, 1.12-1.16]), followed by emergency department visits (RRR, 1.08 [95% CI, 1.04-1.12]). COVID-19-associated utilization for 18 PCCs remained elevated 6 months from the acute stage of infection, with the largest increase in COVID-19-associated utilization observed for infectious disease sequelae (RRR, 86.00 [95% CI, 5.07-1458.33]), COVID-19 (RRR, 19.47 [95% CI, 10.47-36.22]), alopecia (RRR, 2.52 [95% CI, 2.17-2.92]), bronchitis (RRR, 1.85 [95% CI, 1.62-2.12]), pulmonary embolism or deep vein thrombosis (RRR, 1.74 [95% CI, 1.36-2.23]), and dyspnea (RRR, 1.73 [95% CI, 1.61-1.86]). In total, COVID-19-associated excess health care utilization amounted to an estimated 27 217 additional medical encounters over 6 months (212.9 [95% CI, 146.5-278.4] visits per 1000 patients). Conclusions and Relevance: This cohort study documented an excess health care burden of PCC in the 6 months after the acute stage of infection. As health care systems evolve during a highly dynamic and ongoing global pandemic, these data provide valuable evidence to inform long-term strategic resource allocation for patients previously infected with SARS-CoV-2.

Risk of myocarditis and pericarditis following BNT162b2 and mRNA-1273 COVID-19 vaccination.

BACKGROUND: Evidence indicates that mRNA COVID-19 vaccination is associated with risk of myocarditis and possibly pericarditis, especially in young males. It is not clear if risk differs between mRNA-1273 versus BNT162b2. We assessed if risk differs using comprehensive health records on a diverse population. METHODS: Members 18-39 years of age at eight integrated healthcare-delivery systems were monitored using data updated weekly and supplemented with medical record review of myocarditis and pericarditis cases. Incidence of myocarditis and pericarditis events that occurred among vaccine recipients 0 to 7 days after either dose 1 or 2 of a messenger RNA (mRNA) vaccine was compared with that of vaccinated concurrent comparators who, on the same calendar day, had received their most recent dose 22 to 42 days earlier. Rate ratios (RRs) were estimated by conditional Poisson regression, adjusted for age, sex, race and ethnicity, health plan, and calendar day. Head-to-head comparison directly assessed risk following mRNA-1273 versus BNT162b2 during 0-7 days post-vaccination. RESULTS: From December 14, 2020 - January 15, 2022 there were 41 cases after 2,891,498 doses of BNT162b2 and 38 cases after 1,803,267 doses of mRNA-1273. Cases had similar demographic and clinical characteristics. Most were hospitalized for ≤1 day; none required intensive care. During days 0-7 after dose 2 of BNT162b2, the incidence was 14.3 (CI: 6.5-34.9) times higher than the comparison interval, amounting to 22.4 excess cases per million doses; after mRNA-1273 the incidence was 18.8 (CI: 6.7-64.9) times higher than the comparison interval, amounting to 31.2 excess cases per million doses. In head-to-head comparisons 0-7 days after either dose, risk was moderately higher after mRNA-1273 than after BNT162b2 (RR: 1.61, CI 1.02-2.54). CONCLUSIONS: Both vaccines were associated with increased risk of myocarditis and pericarditis in 18-39-year-olds. Risk estimates were modestly higher after mRNA-1273 than after BNT162b2.

Incidence of Guillain-Barré Syndrome After COVID-19 Vaccination in the Vaccine Safety Datalink.

Importance: Postauthorization monitoring of vaccines in a large population may detect rare adverse events not identified in clinical trials such as Guillain-Barré syndrome (GBS), which has a background rate of 1 to 2 per 100 000 person-years. Objective: To describe cases and incidence of GBS following COVID-19 vaccination and assess the risk of GBS after vaccination for Ad.26.COV2.S (Janssen) and mRNA vaccines. Design, Setting, and Participants: This cohort study used surveillance data from the Vaccine Safety Datalink at 8 participating integrated health care systems in the United States. There were 10 158 003 participants aged at least 12 years. Data analysis was performed from November 2021 to February 2022. Exposures: Ad.26.COV2.S, BNT162b2 (Pfizer-BioNTech), or mRNA-1273 (Moderna) COVID-19 vaccine, including mRNA vaccine doses 1 and 2, December 13, 2020, to November 13, 2021. Main Outcomes and Measures: GBS with symptom onset in the 1 to 84 days after vaccination, confirmed by medical record review and adjudication. Descriptive characteristics of confirmed cases, GBS incidence rates during postvaccination risk intervals after each type of vaccine compared with the background rate, rate ratios (RRs) comparing GBS incidence in the 1 to 21 vs 22 to 42 days postvaccination, and RRs directly comparing risk of GBS after Ad.26.COV2.S vs mRNA vaccination, using Poisson regression adjusted for age, sex, race and ethnicity, site, and calendar day. Results: From December 13, 2020, through November 13, 2021, 15 120 073 doses of COVID-19 vaccines were administered to 7 894 989 individuals (mean [SE] age, 46.5 [0.02] years; 8 138 318 doses received [53.8%] by female individuals; 3 671 199 doses received [24.3%] by Hispanic or Latino individuals, 2 215 064 doses received [14.7%] by Asian individuals, 6 266 424 doses received [41.4%] by White individuals), including 483 053 Ad.26.COV2.S doses, 8 806 595 BNT162b2 doses, and 5 830 425 mRNA-1273 doses. Eleven cases of GBS after Ad.26.COV2.S were confirmed. The unadjusted incidence rate of GBS per 100 000 person-years in the 1 to 21 days after Ad.26.COV2.S was 32.4 (95% CI, 14.8-61.5), significantly higher than the background rate, and the adjusted RR in the 1 to 21 vs 22 to 42 days following Ad.26.COV2.S was 6.03 (95% CI, 0.79-147.79). Thirty-six cases of GBS after mRNA vaccines were confirmed. The unadjusted incidence rate per 100 000 person-years in the 1 to 21 days after mRNA vaccines was 1.3 (95% CI, 0.7-2.4) and the adjusted RR in the 1 to 21 vs 22 to 42 days following mRNA vaccines was 0.56 (95% CI, 0.21-1.48). In a head-to-head comparison of Ad.26.COV2.S vs mRNA vaccines, the adjusted RR was 20.56 (95% CI, 6.94-64.66). Conclusions and Relevance: In this cohort study of COVID-19 vaccines, the incidence of GBS was elevated after receiving the Ad.26.COV2.S vaccine. Surveillance is ongoing.

Identification and Validation of Anaphylaxis Using Electronic Health Data in a Population-based Setting.

BACKGROUND: Anaphylaxis is a life-threatening allergic reaction that is difficult to identify accurately with administrative data. We conducted a population-based validation study to assess the accuracy of ICD-10 diagnosis codes for anaphylaxis in outpatient, emergency department, and inpatient settings. METHODS: In an integrated healthcare system in Washington State, we obtained medical records from healthcare encounters with anaphylaxis diagnosis codes (potential events) from October 2015 to December 2018. To capture events missed by anaphylaxis diagnosis codes, we also obtained records on a sample of serious allergic and drug reactions. Two physicians determined whether potential events met established clinical criteria for anaphylaxis (validated events). RESULTS: Out of 239 potential events with anaphylaxis diagnosis codes, the overall positive predictive value (PPV) for validated events was 64% (95% CI = 58 to 70). The PPV decreased with increasing age. Common precipitants for anaphylaxis were food (39%), medications (35%), and insect bite or sting (12%). The sensitivity of emergency department and inpatient anaphylaxis diagnosis codes for all validated events was 58% (95% CI = 51 to 65), but sensitivity increased to 95% (95% CI = 74 to 99) when outpatient diagnosis codes were included. Using information from all validated events and sampling weights, the incidence rate for anaphylaxis was 3.6 events per 10,000 person-years (95% CI = 3.1 to 4.0). CONCLUSIONS: In this population-based setting, ICD-10 diagnosis codes for anaphylaxis from emergency department and inpatient settings had moderate PPV and sensitivity for validated events. These findings have implications for epidemiologic studies that seek to estimate risks of anaphylaxis using electronic health data.

Angiotensin-converting enzyme inhibitor use and pneumonia risk in community-dwelling older adults: results from a population-based case-control study.

PURPOSE: To test whether angiotensin-converting enzyme (ACE) inhibitor use is associated with decreased risk of community-acquired pneumonia in older adults. METHODS: We analyzed data from a nested case-control study of community-dwelling, immunocompetent adults aged 65-94 within an integrated healthcare delivery system. Cases of ambulatory and hospitalized pneumonia from 2000 to 2003 were identified from International Classification of Disease, version 9, codes and validated using medical record review. Controls were matched to cases by age, sex, and calendar year. Using health plan pharmacy data, we defined current use as filling ≥2 prescriptions during the 180 days prior to the case's diagnosis date. We calculated standardized doses per day using World Health Organization defined daily doses. Multivariable conditional logistic regression estimated adjusted odds ratios (ORs) for pneumonia in relation to ACE inhibitor use, adjusting for comorbidity, functional and cognitive status, and other covariates from medical record review and pharmacy data. RESULTS: Current use of ACE inhibitors was seen in 23% (242/1039) of cases and 21% (433/2022) of controls. Lisinopril accounted for 95% of prescriptions. The OR for pneumonia comparing current use to no current use was 0.99 (95% confidence interval [CI] 0.83-1.19). The OR for use of more than two standardized daily doses per day was 1.39 (95% CI 0.93-2.06) compared to no current use. CONCLUSIONS: ACE inhibitor use is not associated with reduced pneumonia risk in community-dwelling older adults.

Use of opioids or benzodiazepines and risk of pneumonia in older adults: a population-based case-control study.

OBJECTIVES: To examine whether use of opioids or benzodiazepines is associated with risk of community-acquired pneumonia in older adults. DESIGN: Population-based case-control study. SETTING: An integrated healthcare delivery system. PARTICIPANTS: Community-dwelling, immunocompetent adults aged 65 to 94 from 2000 to 2003. Presumptive pneumonia cases were identified from health plan automated data and validated through medical record review. Two controls were selected for each case with pneumonia, matched on age, sex, and calendar year. MEASUREMENTS: Information about opioid and benzodiazepine use came from computerized pharmacy data. Information on covariates including comorbid illnesses and functional and cognitive status came from medical record review and electronic health data. RESULTS: One thousand thirty-nine validated cases of pneumonia and 2,022 matched controls were identified. One hundred forty-four (13.9%) cases and 161 (8.0%) controls used prescription opioids (adjusted odds ratio (OR) = 1.38, 95% confidence interval (CI) = 1.08-1.76 vs nonuse). Risk was highest for opioids categorized as immunosuppressive based on immunological studies (OR = 1.88, 95% CI = 1.26-1.79 vs nonuse), whereas for nonimmunosuppressive opioids the OR was 1.23 (95% CI = 0.89-1.69). Risk was highest in the first 14 days of use (OR = 3.24, 95% CI = 1.64-6.39 vs nonuse). For long-acting opioids, the OR was 3.43 (95% CI = 1.44-8.21) versus nonuse, whereas for short-acting opioids, it was 1.27 (95% CI = 0.98-1.64). No greater risk was seen for current benzodiazepine use compared to nonuse (OR = 1.08, 95% CI = 0.80-1.47). CONCLUSION: Use of opioids but not benzodiazepines was associated with pneumonia risk. The differences in risk seen for different opioid regimens warrant further study.

Use of proton pump inhibitors and H2 blockers and risk of pneumonia in older adults: a population-based case-control study.

PURPOSE: To examine whether use of proton pump inhibitors (PPIs) and H2 blockers is associated with increased pneumonia risk. METHODS: We conducted a population-based, nested case-control study within Group Health, an integrated healthcare delivery system. Among community-dwelling, immunocompetent adults aged 65-94, we identified presumptive cases of ambulatory and hospitalized community-acquired pneumonia in 2000-2003 from ICD-9 codes and validated them by medical record review (N = 1125). Controls were selected, matched to cases on age, sex, and calendar year (N = 2235). Current PPI or H2 blocker use was ascertained from computerized pharmacy records. Comorbid illnesses and other characteristics were ascertained by medical record review. Multivariable conditional logistic regression was used to examine the association between medication use and pneumonia risk. We conducted sensitivity analyses using only administrative and pharmacy data to assess how these results differed from our primary results. RESULTS: The prevalence of PPI or H2 blocker use was 21% (241/1125) for pneumonia cases and 16% (350/2235) for controls (adjusted odds ratio [OR] 1.03, 95% CI 0.86-1.24, compared to nonuse). No increased risk was seen for recent initiation. The prevalence of PPI use was 12% (132/1125) for cases and 7% (160/2235) for controls (adjusted OR 1.13, 95% CI 0.88-1.44). Analyses using only administrative and pharmacy data yielded risk estimates farther from the null (adjusted OR 1.32, 95% CI 1.17-1.49, for current PPI use versus nonuse). CONCLUSIONS: Use of PPIs and H2 blockers is not associated with increased pneumonia risk in older adults. The increased risk observed in some prior studies may reflect confounding.

Statin use and risk of community acquired pneumonia in older people: population based case-control study.

OBJECTIVE: To test the hypothesis that hydroxymethyl glutaryl coenzyme A reductase inhibitors (statins) may decrease the risk of community acquired pneumonia. DESIGN: Population based case-control study. SETTING: Group Health, a large integrated healthcare delivery system. Population Immunocompetent, community dwelling Group Health members aged 65 to 94; two matched controls for each case with pneumonia. Information on comorbid illnesses and functional and cognitive status, potential confounders of the association between statin use and risk of pneumonia, came from medical record review and computerised pharmacy data. MAIN OUTCOME MEASURE: Adjusted estimates of risk of pneumonia in relation to current statin use. RESULTS: 1125 validated cases of pneumonia and 2235 matched controls were identified. Compared with controls, cases were more likely to have chronic lung and heart disease, especially severe disease, and functional or cognitive impairment. Current statin use was present in 16.1% (181/1125) of cases and 14.6% (327/2235) of controls (adjusted odds ratio 1.26, 95% confidence interval 1.01 to 1.56). Among cases admitted to hospital and matched controls, current statin use was present in 17.2% (68/395) of cases and 14.2% (112/788) of controls (adjusted odds ratio 1.61, 1.08 to 2.39, compared with non-use). In people in whom statins were indicated for secondary prevention, the adjusted odds ratio for risk of pneumonia in relation to current statin use was 1.25 (0.94 to 1.67); in those with no such indication, it was 0.81 (0.46 to 1.42). CONCLUSIONS: Statin use was not associated with decreased risk of pneumonia among immunocompetent, community dwelling older people. Findings of previous studies may reflect "healthy user" bias.

Vaccines and changes in coagulation parameters in adults on chronic warfarin therapy: a cohort study.

PURPOSE: Warfarin is commonly used among patients who receive influenza, pneumococcal, and tetanus and diphtheria toxoid vaccines, and persons on warfarin therapy may also receive Hepatitis A vaccine. There has been concern that vaccinations could potentially alter coagulation parameters in patients on warfarin therapy. We sought to determine whether vaccinations are associated with changes in International Normalized Ratio (INR) in persons on long-term warfarin therapy. METHODS: We conducted a retrospective cohort study of 5167 members of Group Health, a health maintenance organization (HMO) in western Washington State, who were aged 18 years and older and who were on stable long-term warfarin therapy between 1 January 1992 and 31 December 2003. We made within-person comparisons between mean INR values in the 28 days after receipt of influenza, pneumococcal, tetanus, or hepatitis A vaccine versus mean INR values during other times. RESULTS: Receipt of influenza vaccine was not associated with a change in INR value (mean change, 0.01; 95% confidence interval (CI) -0.01 to 0.03); similar results were observed for pneumococcal (mean change 0.01; 95%CI -0.07 to 0.09), tetanus (mean change 0.03; 95%CI -0.03 to 0.10), and hepatitis A vaccines (mean change 0.03; 95%CI -0.10 to 0.14). CONCLUSIONS: Our results do not suggest that vaccinations lead to clinically significant alterations in coagulation measures among adults on chronic warfarin therapy.