Using machine learning to improve anaphylaxis case identification in medical claims data.

Objectives: Anaphylaxis is a severe life-threatening allergic reaction, and its accurate identification in healthcare databases can harness the potential of "Big Data" for healthcare or public health purposes. Materials and methods: This study used claims data obtained between October 1, 2015 and February 28, 2019 from the CMS database to examine the utility of machine learning in identifying incident anaphylaxis cases. We created a feature selection pipeline to identify critical features between different datasets. Then a variety of unsupervised and supervised methods were used (eg, Sammon mapping and eXtreme Gradient Boosting) to train models on datasets of differing data quality, which reflects the varying availability and potential rarity of ground truth data in medical databases. Results: Resulting machine learning model accuracies ranged from 47.7% to 94.4% when tested on ground truth data. Finally, we found new features to help experts enhance existing case-finding algorithms. Discussion: Developing precise algorithms to detect medical outcomes in claims can be a laborious and expensive process, particularly for conditions presented and coded diversely. We found it beneficial to filter out highly potent codes used for data curation to identify underlying patterns and features. To improve rule-based algorithms where necessary, researchers could use model explainers to determine noteworthy features, which could then be shared with experts and included in the algorithm. Conclusion: Our work suggests machine learning models can perform at similar levels as a previously published expert case-finding algorithm, while also having the potential to improve performance or streamline algorithm construction processes by identifying new relevant features for algorithm construction.

Methodology for Good Machine Learning with Multi-Omics Data.

In 2020, Novartis Pharmaceuticals Corporation and the U.S. Food and Drug Administration (FDA) started a 4-year scientific collaboration to approach complex new data modalities and advanced analytics. The scientific question was to find novel radio-genomics-based prognostic and predictive factors for HR+/HER- metastatic breast cancer under a Research Collaboration Agreement. This collaboration has been providing valuable insights to help successfully implement future scientific projects, particularly using artificial intelligence and machine learning. This tutorial aims to provide tangible guidelines for a multi-omics project that includes multidisciplinary expert teams, spanning across different institutions. We cover key ideas, such as "maintaining effective communication" and "following good data science practices," followed by the four steps of exploratory projects, namely (1) plan, (2) design, (3) develop, and (4) disseminate. We break each step into smaller concepts with strategies for implementation and provide illustrations from our collaboration to further give the readers actionable guidance.

Using machine learning to improve anaphylaxis case identification in medical claims data.

Objective: Anaphylaxis is a severe life-threatening allergic reaction, and its accurate identification in healthcare databases can harness the potential of "Big Data" for healthcare or public health purposes. Methods: This study used claims data obtained between October 1, 2015 and February 28, 2019 from the CMS database to examine the utility of machine learning in identifying incident anaphylaxis cases. We created a feature selection pipeline to identify critical features between different datasets. Then a variety of unsupervised and supervised methods were used (eg, Sammon mapping and eXtreme Gradient Boosting) to train models on datasets of differing data quality, which reflects the varying availability and potential rarity of ground truth data in medical databases. Results: Resulting machine learning model accuracies ranged between 47.7% and 94.4% when tested on ground truth data. Finally, we found new features to help experts enhance existing case-finding algorithms. Discussion: Developing precise algorithms to detect medical outcomes in claims can be a laborious and expensive process, particularly for conditions presented and coded diversely. We found it beneficial to filter out highly potent codes used for data curation to identify underlying patterns and features. To improve rule-based algorithms where necessary, researchers could use model explainers to determine noteworthy features, which could then be shared with experts and included in the algorithm. Conclusion: Our work suggests machine learning models can perform at similar levels as a previously published expert case-finding algorithm, while also having the potential to improve performance or streamline algorithm construction processes by identifying new relevant features for algorithm construction.

Communicating regulatory high-throughput sequencing data using BioCompute Objects.

This project demonstrates the use of the IEEE 2791-2020 Standard (BioCompute Objects [BCO]) to enable the complete and concise communication of results from next generation sequencing (NGS) analysis. One arm of a clinical trial was replicated using synthetically generated data made to resemble real biological data and then two independent analyses were performed. The first simulated a pharmaceutical regulatory submission to the US Food and Drug Administration (FDA) including analysis of results and a BCO. The second simulated an FDA review that included an independent analysis of the submitted data. Of the 118 simulated patient samples generated, 117 (99.15%) were in agreement in the two analyses. This process exemplifies how a template BCO (tBCO), including a verification kit, facilitates transparency and reproducibility, thereby reinforcing confidence in the regulatory submission process.

Use of a rapid electronic survey methodology to estimate blood donors' potential exposure to emerging infectious diseases: Application of a statistically representative sampling methodology to assess risk in US blood centers.

Risk assessments of transfusion-transmitted emerging infectious diseases (EIDs) are complicated by the fact that blood donors' demographics and behaviors can be different from the general population. Therefore, when assessing potential blood donor exposure to EIDs, the use of general population characteristics, such as U.S. travel statistics, may invoke uncertainties that result in inaccurate estimates of blood donor exposure. This may, in turn, lead to the creation of donor deferral policies that do not match actual risk. STUDY DESIGN AND METHODS: This article reports on the development of a system to rapidly assess EID risks for a nationally representative portion of the U.S. blood donor population. To assess the effectiveness of this system, a test survey was developed and deployed to a statistically representative sample frame of blood donors from five blood collecting organizations. Donors were directed to an online survey to ascertain their recent travel and potential exposure to Middle East respiratory syndrome coronavirus (MERS-CoV). RESULTS: A total of 7128 responses were received from 54 256 invitations. The age-adjusted estimated total number of blood donors potentially exposed to MERS-CoV was approximately 15 640 blood donors compared to a lower U.S. general population-based estimate of 9610 blood donors. CONCLUSION: The structured donor demographic sample-based data provided an assessment of blood donors' potential exposure to an emerging pathogen that was 63% larger than the U.S. population-based estimate. This illustrates the need for tailored blood donor-based EID risk assessments that provide more specific demographic risk intelligence and can inform appropriate regulatory decision making.

An Inter-regional US Blood Supply Simulation Model to Evaluate Blood Availability to Support Planning for Emergency Preparedness and Medical Countermeasures.

OBJECTIVES: Planning for a response to threats like pandemics or mass casualty events is a national priority. The US blood supply system can be particularly vulnerable to such events. It is important to understand the impacts of emergency situations on blood availability and the resiliency of the US blood supply system. METHODS: On the basis of the Stock-and-Flow simulation model of the US blood supply system, we developed an inter-regional blood transfer system representing the action of multiple blood collectors and distributors to enable effective planning of strategies to minimize collection and donation disruptions to the blood supply system in the event of a national emergency. RESULTS: We simulated a pandemic or mass casualty event on both a national and an inter-regional blood supply system. Differences in the estimated impacts demonstrated the importance of incorporating spatial and temporal variations of blood collection and utilization across US regions. The absence of blood shortage in both emergency scenarios highlighted the resilience of the inter-regional system to meet the potential associated blood demand. CONCLUSIONS: Our inter-regional model considered complex factors and can be a valuable tool to assist regulatory decision-making and strategic planning for emergency preparedness to avoid and mitigate associated adverse health consequences. (Disaster Med Public Health Preparedness. 2018;12:201-210).

Enabling precision medicine via standard communication of HTS provenance, analysis, and results.

A personalized approach based on a patient's or pathogen's unique genomic sequence is the foundation of precision medicine. Genomic findings must be robust and reproducible, and experimental data capture should adhere to findable, accessible, interoperable, and reusable (FAIR) guiding principles. Moreover, effective precision medicine requires standardized reporting that extends beyond wet-lab procedures to computational methods. The BioCompute framework (https://w3id.org/biocompute/1.3.0) enables standardized reporting of genomic sequence data provenance, including provenance domain, usability domain, execution domain, verification kit, and error domain. This framework facilitates communication and promotes interoperability. Bioinformatics computation instances that employ the BioCompute framework are easily relayed, repeated if needed, and compared by scientists, regulators, test developers, and clinicians. Easing the burden of performing the aforementioned tasks greatly extends the range of practical application. Large clinical trials, precision medicine, and regulatory submissions require a set of agreed upon standards that ensures efficient communication and documentation of genomic analyses. The BioCompute paradigm and the resulting BioCompute Objects (BCOs) offer that standard and are freely accessible as a GitHub organization (https://github.com/biocompute-objects) following the "Open-Stand.org principles for collaborative open standards development." With high-throughput sequencing (HTS) studies communicated using a BCO, regulatory agencies (e.g., Food and Drug Administration [FDA]), diagnostic test developers, researchers, and clinicians can expand collaboration to drive innovation in precision medicine, potentially decreasing the time and cost associated with next-generation sequencing workflow exchange, reporting, and regulatory reviews.

Natural language processing systems for capturing and standardizing unstructured clinical information: A systematic review.

We followed a systematic approach based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses to identify existing clinical natural language processing (NLP) systems that generate structured information from unstructured free text. Seven literature databases were searched with a query combining the concepts of natural language processing and structured data capture. Two reviewers screened all records for relevance during two screening phases, and information about clinical NLP systems was collected from the final set of papers. A total of 7149 records (after removing duplicates) were retrieved and screened, and 86 were determined to fit the review criteria. These papers contained information about 71 different clinical NLP systems, which were then analyzed. The NLP systems address a wide variety of important clinical and research tasks. Certain tasks are well addressed by the existing systems, while others remain as open challenges that only a small number of systems attempt, such as extraction of temporal information or normalization of concepts to standard terminologies. This review has identified many NLP systems capable of processing clinical free text and generating structured output, and the information collected and evaluated here will be important for prioritizing development of new approaches for clinical NLP.

Evaluation of Facebook and Twitter Monitoring to Detect Safety Signals for Medical Products: An Analysis of Recent FDA Safety Alerts.

INTRODUCTION: The rapid expansion of the Internet and computing power in recent years has opened up the possibility of using social media for pharmacovigilance. While this general concept has been proposed by many, central questions remain as to whether social media can provide earlier warnings for rare and serious events than traditional signal detection from spontaneous report data. OBJECTIVE: Our objective was to examine whether specific product-adverse event pairs were reported via social media before being reported to the US FDA Adverse Event Reporting System (FAERS). METHODS: A retrospective analysis of public Facebook and Twitter data was conducted for 10 recent FDA postmarketing safety signals at the drug-event pair level with six negative controls. Social media data corresponding to two years prior to signal detection of each product-event pair were compiled. Automated classifiers were used to identify each 'post with resemblance to an adverse event' (Proto-AE), among English language posts. A custom dictionary was used to translate Internet vernacular into Medical Dictionary for Regulatory Activities (MedDRA®) Preferred Terms. Drug safety physicians conducted a manual review to determine causality using World Health Organization-Uppsala Monitoring Centre (WHO-UMC) assessment criteria. Cases were also compared with those reported in FAERS. FINDINGS: A total of 935,246 posts were harvested from Facebook and Twitter, from March 2009 through October 2014. The automated classifier identified 98,252 Proto-AEs. Of these, 13 posts were selected for causality assessment of product-event pairs. Clinical assessment revealed that posts had sufficient information to warrant further investigation for two possible product-event associations: dronedarone-vasculitis and Banana Boat Sunscreen--skin burns. No product-event associations were found among the negative controls. In one of the positive cases, the first report occurred in social media prior to signal detection from FAERS, whereas the other case occurred first in FAERS. CONCLUSIONS: An efficient semi-automated approach to social media monitoring may provide earlier insights into certain adverse events. More work is needed to elaborate additional uses for social media data in pharmacovigilance and to determine how they can be applied by regulatory agencies.

Decision support environment for medical product safety surveillance.

We have developed a Decision Support Environment (DSE) for medical experts at the US Food and Drug Administration (FDA). The DSE contains two integrated systems: The Event-based Text-mining of Health Electronic Records (ETHER) and the Pattern-based and Advanced Network Analyzer for Clinical Evaluation and Assessment (PANACEA). These systems assist medical experts in reviewing reports submitted to the Vaccine Adverse Event Reporting System (VAERS) and the FDA Adverse Event Reporting System (FAERS). In this manuscript, we describe the DSE architecture and key functionalities, and examine its potential contributions to the signal management process by focusing on four use cases: the identification of missing cases from a case series, the identification of duplicate case reports, retrieving cases for a case series analysis, and community detection for signal identification and characterization.

Babesiosis Occurrence among the Elderly in the United States, as Recorded in Large Medicare Databases during 2006-2013.

BACKGROUND: Human babesiosis, caused by intraerythrocytic protozoan parasites, can be an asymptomatic or mild-to-severe disease that may be fatal. The study objective was to assess babesiosis occurrence among the U.S. elderly Medicare beneficiaries, ages 65 and older, during 2006-2013. METHODS: Our retrospective claims-based study utilized large Medicare administrative databases. Babesiosis occurrence was ascertained by recorded ICD-9-CM diagnosis code. The study assessed babesiosis occurrence rates (per 100,000 elderly Medicare beneficiaries) overall and by year, age, gender, race, state of residence, and diagnosis months. RESULTS: A total of 10,305 elderly Medicare beneficiaries had a recorded babesiosis diagnosis during the eight-year study period, for an overall rate of about 5 per 100,000 persons. Study results showed a significant increase in babesiosis occurrence over time (p<0.05), with the largest number of cases recorded in 2013 (N = 1,848) and the highest rates (per 100,000) in five Northeastern states: Connecticut (46), Massachusetts (45), Rhode Island (42), New York (27), and New Jersey (14). About 75% of all cases were diagnosed from May through October. Babesiosis occurrence was significantly higher among males vs. females and whites vs. non-whites. CONCLUSION: Our study reveals increasing babesiosis occurrence among the U.S. elderly during 2006-2013, with highest rates in the babesiosis-endemic states. The study also shows variation in babesiosis occurrence by age, gender, race, state of residence, and diagnosis months. Overall, our study highlights the importance of large administrative databases in assessing the occurrence of emerging infections in the United States.

A comparative pharmacokinetic estimate of mercury in U.S. Infants following yearly exposures to inactivated influenza vaccines containing thimerosal.

The use of thimerosal preservative in childhood vaccines has been largely eliminated over the past decade in the United States because vaccines have been reformulated in single-dose vials that do not require preservative. An exception is the inactivated influenza vaccines, which are formulated in both multidose vials requiring preservative and preservative-free single-dose vials. As part of an ongoing evaluation by USFDA of the safety of biologics throughout their lifecycle, the infant body burden of mercury following scheduled exposures to thimerosal preservative in inactivated influenza vaccines in the United States was estimated and compared to the infant body burden of mercury following daily exposures to dietary methylmercury at the reference dose established by the USEPA. Body burdens were estimated using kinetic parameters derived from experiments conducted in infant monkeys that were exposed episodically to thimerosal or MeHg at identical doses. We found that the body burden of mercury (AUC) in infants (including low birth weight) over the first 4.5 years of life following yearly exposures to thimerosal was two orders of magnitude lower than that estimated for exposures to the lowest regulatory threshold for MeHg over the same time period. In addition, peak body burdens of mercury following episodic exposures to thimerosal in this worst-case analysis did not exceed the corresponding safe body burden of mercury from methylmercury at any time, even for low-birth-weight infants. Our pharmacokinetic analysis supports the acknowledged safety of thimerosal when used as a preservative at current levels in certain multidose infant vaccines in the United States.

A stock-and-flow simulation model of the US blood supply.

BACKGROUND: Lack of reporting requirements for the amount of blood stored in blood banks and hospitals poses challenges to effectively monitor the US blood supply. Effective strategies to minimize collection and donation disruptions in the supply require an understanding of the daily amount of blood available in the system. STUDY DESIGN AND METHODS: A stock-and-flow simulation model of the US blood supply was developed to obtain estimates of the daily on-hand availability of blood, with uncertainty and by ABO/Rh type. The model simulated potential impact on supply of using different blood management practices for transfusion: first in-first out (FIFO), using the oldest stored red blood cell units first; non-FIFO likely oldest, preferentially selecting older blood; and non-FIFO likely newest, preferentially selecting younger blood. RESULTS: Simulation results showed higher estimates of the steady-state of the blood supply level for FIFO (1,630,000 units, 95% prediction interval [PI] 1,610,000-1,650,000) than non-FIFO scenarios (likely oldest, 1,530,000 units, 95% PI 1,500,000-1,550,000; and likely newest, 1,190,000 units, 95% PI 1,160,000-1,220,000), either for overall blood or by blood types. CONCLUSION: To our knowledge, this model represents a first attempt to evaluate the impact of different blood management practices on daily availability and distribution of blood in the US blood supply. The average storage time before blood is being issued was influenced by blood management practices, for preferences of blood that is younger and also that use specific blood types. The model also suggests which practice could best approximate the current blood management system and may serve as useful tool for blood management.

Babesiosis among elderly Medicare beneficiaries, United States, 2006-2008.

We used administrative databases to assess babesiosis among elderly persons in the United States by year, sex, age, race, state of residence, and diagnosis months during 2006-2008. The highest babesiosis rates were in Connecticut, Rhode Island, New York, and Massachusetts, and findings suggested babesiosis expansion to other states.

Updated aluminum pharmacokinetics following infant exposures through diet and vaccination.

Aluminum is a ubiquitous element that is released naturally into the environment via volcanic activity and the breakdown of rocks on the earth's surface. Exposure of the general population to aluminum occurs primarily through the consumption of food, antacids, and buffered analgesics. Exposure to aluminum in the general population can also occur through vaccination, since vaccines often contain aluminum salts (frequently aluminum hydroxide or aluminum phosphate) as adjuvants. Because concerns have been expressed by the public that aluminum in vaccines may pose a risk to infants, we developed an up-to-date analysis of the safety of aluminum adjuvants. Keith et al. [1] previously analyzed the pharmacokinetics of aluminum for infant dietary and vaccine exposures and compared the resulting body burdens to those based on the minimal risk levels (MRLs) established by the Agency for Toxic Substances and Disease Registry. We updated the analysis of Keith et al. [1] with a current pediatric vaccination schedule [2]; baseline aluminum levels at birth; an aluminum retention function that reflects changing glomerular filtration rates in infants; an adjustment for the kinetics of aluminum efflux at the site of injection; contemporaneous MRLs; and the most recent infant body weight data for children 0-60 months of age [3]. Using these updated parameters we found that the body burden of aluminum from vaccines and diet throughout an infant's first year of life is significantly less than the corresponding safe body burden of aluminum modeled using the regulatory MRL. We conclude that episodic exposures to vaccines that contain aluminum adjuvant continue to be extremely low risk to infants and that the benefits of using vaccines containing aluminum adjuvant outweigh any theoretical concerns.

Statistical, epidemiological, and risk-assessment approaches to evaluating safety of vaccines throughout the life cycle at the Food and Drug Administration.

The public health community faces increasing demands for improving vaccine safety while simultaneously increasing the number of vaccines available to prevent infectious diseases. The passage of the US Food and Drug Administration (FDA) Amendment Act of 2007 formalized the concept of life-cycle management of the risks and benefits of vaccines, from early clinical development through many years of use in large numbers of people. Harnessing scientific and technologic advances is necessary to improve vaccine-safety evaluation. The Office of Biostatistics and Epidemiology in the Center for Biologics Evaluation and Research is working to improve the FDA's ability to monitor vaccine safety by improving statistical, epidemiologic, and risk-assessment methods, gaining access to new sources of data, and exploring the use of genomics data. In this article we describe the current approaches, new resources, and future directions that the FDA is taking to improve the evaluation of vaccine safety.

Risk assessment and cost-effectiveness/utility analysis.

Decision-makers at all levels of public health and transfusion medicine have always assessed the risks and benefits of their decisions. Decisions are usually guided by immediately available information and a significant amount of experience and judgment. For decisions concerning familiar situations and common problems, judgment and experience may work quite well, but this type of decision process can lack clarity and accountability. Public health challenges are changing as emerging diseases and expensive technologies complicate the decision-makers' task, confronting the decision-maker with new problems that include multiple potential solutions. Decisions regarding policies and adoption of technologies are particularly complex in transfusion medicine due to the scope of the field, implications for public health, and legal, regulatory and public expectations regarding blood safety. To assist decision-makers, quantitative risk assessment and cost-effectiveness analysis are now being more widely applied. This set of articles will introduce risk assessment and cost-effectiveness methodologies and discuss recent applications of these methods in transfusion medicine.

Potential for internalization, growth, and survival of Salmonella and Escherichia coli O157:H7 in oranges.

Internalization potential, survival, and growth of human pathogens within oranges were investigated in a series of laboratory experiments. Submerging oranges into dye solutions at various temperature differentials was used to assess internalization potential. Conditions in which dye internalization was observed were further studied by applying Escherichia coli O157:H7 or Salmonella onto the stem scar, subjecting the oranges to a temperature differential, juicing, and measuring numbers of pathogens in the resulting juice. Pathogens for growth and survival studies were applied to or injected into simulated peel punctures. Oranges with small peel holes of selected sizes were also placed into solutions containing these pathogens. Bacterial survival was also evaluated in orange juice at 4 and 24 degrees C. Oranges internalized pathogens at a frequency of 2.5 to 3.0%, which mirrored dye internalization frequency (3.3%). Pathogens were internalized at an uptake level of 0.1 to 0.01% of the challenge applied. Bacteria grew within oranges at 24 degrees C, but not at 4 degrees C. Thirty-one percent of oranges with 0.91-mm surface holes showed pathogen uptake, whereas 2% of oranges with 0.68-mm holes showed pathogen uptake. Pathogens added to fresh orange juice and incubated at 24 degrees C declined 1 log CFU/ml within 3 days. These results suggest that internalization, survival, and growth of human bacterial pathogens can occur within oranges intended for producing unpasteurized juice.