Using machine learning to improve anaphylaxis case identification in medical claims data.

Objectives: Anaphylaxis is a severe life-threatening allergic reaction, and its accurate identification in healthcare databases can harness the potential of "Big Data" for healthcare or public health purposes. Materials and methods: This study used claims data obtained between October 1, 2015 and February 28, 2019 from the CMS database to examine the utility of machine learning in identifying incident anaphylaxis cases. We created a feature selection pipeline to identify critical features between different datasets. Then a variety of unsupervised and supervised methods were used (eg, Sammon mapping and eXtreme Gradient Boosting) to train models on datasets of differing data quality, which reflects the varying availability and potential rarity of ground truth data in medical databases. Results: Resulting machine learning model accuracies ranged from 47.7% to 94.4% when tested on ground truth data. Finally, we found new features to help experts enhance existing case-finding algorithms. Discussion: Developing precise algorithms to detect medical outcomes in claims can be a laborious and expensive process, particularly for conditions presented and coded diversely. We found it beneficial to filter out highly potent codes used for data curation to identify underlying patterns and features. To improve rule-based algorithms where necessary, researchers could use model explainers to determine noteworthy features, which could then be shared with experts and included in the algorithm. Conclusion: Our work suggests machine learning models can perform at similar levels as a previously published expert case-finding algorithm, while also having the potential to improve performance or streamline algorithm construction processes by identifying new relevant features for algorithm construction.

Using machine learning to improve anaphylaxis case identification in medical claims data.

Objective: Anaphylaxis is a severe life-threatening allergic reaction, and its accurate identification in healthcare databases can harness the potential of "Big Data" for healthcare or public health purposes. Methods: This study used claims data obtained between October 1, 2015 and February 28, 2019 from the CMS database to examine the utility of machine learning in identifying incident anaphylaxis cases. We created a feature selection pipeline to identify critical features between different datasets. Then a variety of unsupervised and supervised methods were used (eg, Sammon mapping and eXtreme Gradient Boosting) to train models on datasets of differing data quality, which reflects the varying availability and potential rarity of ground truth data in medical databases. Results: Resulting machine learning model accuracies ranged between 47.7% and 94.4% when tested on ground truth data. Finally, we found new features to help experts enhance existing case-finding algorithms. Discussion: Developing precise algorithms to detect medical outcomes in claims can be a laborious and expensive process, particularly for conditions presented and coded diversely. We found it beneficial to filter out highly potent codes used for data curation to identify underlying patterns and features. To improve rule-based algorithms where necessary, researchers could use model explainers to determine noteworthy features, which could then be shared with experts and included in the algorithm. Conclusion: Our work suggests machine learning models can perform at similar levels as a previously published expert case-finding algorithm, while also having the potential to improve performance or streamline algorithm construction processes by identifying new relevant features for algorithm construction.

Risk of Guillain-Barré Syndrome Following Recombinant Zoster Vaccine in Medicare Beneficiaries.

Importance: Guillain-Barré syndrome can be reported after vaccination. This study assesses the risk of Guillain-Barré syndrome after administration of recombinant zoster vaccine (RZV or Shingrix), which is administered in 2 doses 2 to 6 months apart. Objective: Use Medicare claims data to evaluate risk of developing Guillain-Barré syndrome following vaccination with zoster vaccine. Design, Setting, and Participants: This case series cohort study included 849 397 RZV-vaccinated and 1 817 099 zoster vaccine live (ZVL or Zostavax)-vaccinated beneficiaries aged 65 years or older. Self-controlled analyses included events identified from 2 113 758 eligible RZV-vaccinated beneficiaries 65 years or older. We compared the relative risk of Guillain-Barré syndrome after RZV vs ZVL, followed by claims-based and medical record-based self-controlled case series analyses to assess risk of Guillain-Barré syndrome during a postvaccination risk window (days 1-42) compared with a control window (days 43-183). In self-controlled analyses, RZV vaccinees were observed from October 1, 2017, to February 29, 2020. Patients were identified in the inpatient, outpatient procedural (including emergency department), and office settings using Medicare administrative data. Exposures: Vaccination with RZV or ZVL vaccines. Main Outcomes and Measures: Guillain-Barré syndrome was identified in Medicare administrative claims data, and cases were assessed through medical record review using the Brighton Collaboration case definition. Results: Amongst those who received RZV vaccinees, the mean age was 74.8 years at first dose, and 58% were women, whereas among those who received the ZVL vaccine, the mean age was 74.3 years, and 60% were women. In the cohort analysis we detected an increase in risk of Guillain-Barré syndrome among RZV vaccinees compared with ZVL vaccinees (rate ratio [RR], 2.34; 95% CI, 1.01-5.41; P = .047). In the self-controlled analyses, we observed 24 and 20 cases during the risk and control period, respectively. Our claims-based analysis identified an increased risk in the risk window compared with the control window (RR, 2.84; 95% CI, 1.53-5.27; P = .001), with an attributable risk of 3 per million RZV doses (95% CI, 0.62-5.64). Our medical record-based analysis confirmed this increased risk (RR, 4.96; 95% CI, 1.43-17.27; P = .01). Conclusions and Relevance: Findings of this case series cohort study indicate a slightly increased risk of Guillain-Barré syndrome during the 42 days following RZV vaccination in the Medicare population, with approximately 3 excess Guillain-Barré syndrome cases per million vaccinations. Clinicians and patients should be aware of this risk, while considering the benefit of decreasing the risk of herpes zoster and its complications through an efficacious vaccine, as risk-benefit balance remains in favor of vaccination.

ICD-10 anaphylaxis algorithm and the estimate of vaccine-attributable anaphylaxis incidence in Medicare.

BACKGROUND: Anaphylaxis is a rare, serious allergic reaction. Its identification in large healthcare databases can help better characterize this risk. OBJECTIVE: To create an ICD-10 anaphylaxis algorithm, estimate its positive predictive values (PPVs) in a post-vaccination risk window, and estimate vaccination-attributable anaphylaxis rates in the Medicare Fee For Service (FFS) population. METHODS: An anaphylaxis algorithm with core and extended portions was constructed analyzing ICD-10 anaphylaxis claims data in Medicare FFS from 2015 to 2017. Cases of post-vaccination anaphylaxis among Medicare FFS beneficiaries were then identified from October 1, 2015 to February 28, 2019 utilizing vaccine relevant anaphylaxis ICD-10 codes. Information from medical records was used to determine true anaphylaxis cases based on the Brighton Collaboration's anaphylaxis case definition. PPVs were estimated for incident anaphylaxis and the subset of vaccine-attributable anaphylaxis within a 2-day post-vaccination risk window. Vaccine-attributable anaphylaxis rates in Medicare FFS were also estimated. RESULTS: The study recorded 66,572,128 vaccinations among 21,685,119 unique Medicare FFS beneficiaries. The algorithm identified a total of 190 suspected anaphylaxis cases within the 2-day post-vaccination window; of these 117 (62%) satisfied the core algorithm, and 73 (38%) additional cases satisfied the extended algorithm. The core algorithm's PPV was 66% (95% CI [56%, 76%]) for identifying incident anaphylaxis and 44% (95% CI [34%, 56%]) for vaccine-attributable anaphylaxis. The vaccine-attributable anaphylaxis incidence rate after any vaccination was 0.88 per million doses (95% CI [0.67, 1.16]). CONCLUSION: The ICD-10 claims algorithm for anaphylaxis allows the assessment of anaphylaxis risk in real-world data. The algorithm revealed vaccine-attributable anaphylaxis is rare among vaccinated Medicare FFS beneficiaries.

Thrombocytopenia including immune thrombocytopenia after receipt of mRNA COVID-19 vaccines reported to the Vaccine Adverse Event Reporting System (VAERS).

BACKGROUND: The objective of this study is to assess cases of thrombocytopenia, including immune thrombocytopenia (ITP), reported to the Vaccine Adverse Event Reporting System (VAERS) following vaccination with mRNA COVID-19 vaccines. METHODS: This case-series study analyzed VAERS reports of thrombocytopenia after vaccination with Pfizer-BioNTech COVID-19 Vaccine or Moderna COVID-19 Vaccine. RESULTS: Fifteen cases of thrombocytopenia were identified among 18,841,309 doses of Pfizer-BioNTech COVID-19 Vaccine and 13 cases among 16,260,102 doses of Moderna COVID-19 Vaccine. The reporting rate of thrombocytopenia was 0.80 per million doses for both vaccines. Based on an annual incidence rate of 3.3 ITP cases per 100,000 adults, the observed number of all thrombocytopenia cases, which includes ITP, following administration of mRNA COVID-19 vaccines is not greater than the number of ITP cases expected. CONCLUSIONS: The number of thrombocytopenia cases reported to VAERS does not suggest a safety concern attributable to mRNA COVID-19 vaccines at this time.

Postlicensure Safety Surveillance of Recombinant Zoster Vaccine (Shingrix) - United States, October 2017-June 2018.

Recombinant zoster vaccine (RZV; Shingrix), an adjuvanted glycoprotein vaccine, was licensed by the Food and Drug Administration (FDA) and recommended by the Advisory Committee on Immunization Practices for adults aged ≥50 years in October 2017 (1). The previously licensed live-attenuated zoster vaccine (ZVL; Zostavax) is recommended for adults aged ≥60 years. RZV is administered intramuscularly as a 2-dose series, with an interval of 2-6 months between doses. In prelicensure clinical trials, 85% of 6,773 vaccinated study participants reported local or systemic reactions after receiving RZV, with approximately 17% experiencing a grade 3 reaction (erythema or induration >3.5 inches or systemic symptoms that interfere with normal activity). However, rates of serious adverse events (i.e., hospitalization, prolongation of existing hospitalization, life-threatening illness, permanent disability, congenital anomaly or birth defect, or death) were similar in the RZV and placebo groups (2). After licensure, CDC and FDA began safety monitoring of RZV in the Vaccine Adverse Event Reporting System (VAERS) (3). During the first 8 months of use, when approximately 3.2 million RZV doses were distributed (GlaxoSmithKline, personal communication, 2018), VAERS received a total of 4,381 reports of adverse events, 130 (3.0%) of which were classified as serious. Commonly reported signs and symptoms included pyrexia (fever) (1,034; 23.6%), injection site pain (985; 22.5%), and injection site erythema (880; 20.1%). No unexpected patterns were detected in reports of adverse events or serious adverse events. Findings from early monitoring of RZV are consistent with the safety profile observed in prelicensure clinical trials.

Decision support environment for medical product safety surveillance.

We have developed a Decision Support Environment (DSE) for medical experts at the US Food and Drug Administration (FDA). The DSE contains two integrated systems: The Event-based Text-mining of Health Electronic Records (ETHER) and the Pattern-based and Advanced Network Analyzer for Clinical Evaluation and Assessment (PANACEA). These systems assist medical experts in reviewing reports submitted to the Vaccine Adverse Event Reporting System (VAERS) and the FDA Adverse Event Reporting System (FAERS). In this manuscript, we describe the DSE architecture and key functionalities, and examine its potential contributions to the signal management process by focusing on four use cases: the identification of missing cases from a case series, the identification of duplicate case reports, retrieving cases for a case series analysis, and community detection for signal identification and characterization.

Novel algorithms for improved pattern recognition using the US FDA Adverse Event Network Analyzer.

The medical review of adverse event reports for medical products requires the processing of "big data" stored in spontaneous reporting systems, such as the US Vaccine Adverse Event Reporting System (VAERS). VAERS data are not well suited to traditional statistical analyses so we developed the FDA Adverse Event Network Analyzer (AENA) and three novel network analysis approaches to extract information from these data. Our new approaches include a weighting scheme based on co-occurring triplets in reports, a visualization layout inspired by the islands algorithm, and a network growth methodology for the detection of outliers. We explored and verified these approaches by analysing the historical signal of Intussusception (IS) after the administration of RotaShield vaccine (RV) in 1999. We believe that our study supports the use of AENA for pattern recognition in medical product safety and other clinical data.