Recommendations for achieving interoperable and shareable medical data in the USA.

Easy access to large quantities of accurate health data is required to understand medical and scientific information in real-time; evaluate public health measures before, during, and after times of crisis; and prevent medical errors. Introducing a system in the USA that allows for efficient access to such health data and ensures auditability of data facts, while avoiding data silos, will require fundamental changes in current practices. Here, we recommend the implementation of standardized data collection and transmission systems, universal identifiers for individual patients and end users, a reference standard infrastructure to support calibration and integration of laboratory results from equivalent tests, and modernized working practices. Requiring comprehensive and binding standards, rather than incentivizing voluntary and often piecemeal efforts for data exchange, will allow us to achieve the analytical information environment that patients need.

Signaling COVID-19 Vaccine Adverse Events.

INTRODUCTION: Statistical signal detection is a crucial tool for rapidly identifying potential risks associated with pharmaceutical products. The unprecedented environment created by the coronavirus disease 2019 (COVID-19) pandemic for vaccine surveillance predisposes commonly applied signal detection methodologies to a statistical issue called the masking effect, in which signals for a vaccine of interest are hidden by the presence of other reported vaccines. This masking effect may in turn limit or delay our understanding of the risks associated with new and established vaccines. OBJECTIVE: The aim is to investigate the problem of masking in the context of COVID-19 vaccine signal detection, assessing its impact, extent, and root causes. METHODS: Based on data underlying the Vaccine Adverse Event Reporting System, three commonly applied statistical signal detection methodologies, and a more advanced regression-based methodology, we investigate the temporal evolution of signals corresponding to five largely recognized adverse events and two potentially new adverse events. RESULTS: The results demonstrate that signals of adverse events related to COVID-19 vaccines may be undetected or delayed due to masking when generated by methodologies currently utilized by pharmacovigilance organizations, and that a class of advanced methodologies can partially alleviate the problem. The results indicate that while masking is rare relative to all possible statistical associations, it is much more likely to occur in COVID-19 vaccine signaling, and that its extent, direction, impact, and roots are not static, but rather changing in accordance with the changing nature of data. CONCLUSIONS: Masking is an addressable problem that merits careful consideration, especially in situations such as COVID-19 vaccine safety surveillance and other emergency use authorization products.

Clinical Concept Value Sets and Interoperability in Health Data Analytics.

This paper focuses on value sets as an essential component in the health analytics ecosystem. We discuss shared repositories of reusable value sets and offer recommendations for their further development and adoption. In order to motivate these contributions, we explain how value sets fit into specific analytic tasks and the health analytics landscape more broadly; their growing importance and ubiquity with the advent of Common Data Models, Distributed Research Networks, and the availability of higher order, reusable analytic resources like electronic phenotypes and electronic clinical quality measures; the formidable barriers to value set reuse; and our introduction of a concept-agnostic orientation to vocabulary collections. The costs of ad hoc value set management and the benefits of value set reuse are described or implied throughout. Our standards, infrastructure, and design recommendations are not systematic or comprehensive but invite further work to support value set reuse for health analytics. The views represented in the paper do not necessarily represent the views of the institutions or of all the co-authors.

Regulatory Forum Review*: Utility of in Vitro Secondary Pharmacology Data to Assess Risk of Drug-induced Valvular Heart Disease in Humans: Regulatory Considerations.

Drug-induced valvular heart disease (VHD) is a serious side effect linked to long-term treatment with 5-hydroxytryptamine (serotonin) receptor 2B (5-HT2B) agonists. Safety assessment for off-target pharmacodynamic activity is a common approach used to screen drugs for this undesired property. Such studies include in vitro assays to determine whether the drug is a 5-HT2B agonist, a necessary pharmacological property for development of VHD. Measures of in vitro binding affinity (IC50, Ki) or cellular functional activity (EC50) are often compared to maximum therapeutic free plasma drug levels ( fCmax) from which safety margins (SMs) can be derived. However, there is no clear consensus on what constitutes an appropriate SM under various therapeutic conditions of use. The strengths and limitations of SM determinations and current risk assessment methodology are reviewed and evaluated. It is concluded that the use of SMs based on Ki values, or those relative to serotonin (5-HT), appears to be a better predictor than the use of EC50 or EC50/human fCmax values for determining whether known 5-HT2B agonists have resulted in VHD. It is hoped that such a discussion will improve efforts to reduce this preventable serious drug-induced toxicity from occurring and lead to more informed risk assessment strategies.

Use of data mining at the Food and Drug Administration.

OBJECTIVES: This article summarizes past and current data mining activities at the United States Food and Drug Administration (FDA). TARGET AUDIENCE: We address data miners in all sectors, anyone interested in the safety of products regulated by the FDA (predominantly medical products, food, veterinary products and nutrition, and tobacco products), and those interested in FDA activities. SCOPE: Topics include routine and developmental data mining activities, short descriptions of mined FDA data, advantages and challenges of data mining at the FDA, and future directions of data mining at the FDA.

Leveraging MEDLINE indexing for pharmacovigilance - Inherent limitations and mitigation strategies.

BACKGROUND: Traditional approaches to pharmacovigilance center on the signal detection from spontaneous reports, e.g., the U.S. Food and Drug Administration (FDA) adverse event reporting system (FAERS). In order to enrich the scientific evidence and enhance the detection of emerging adverse drug events that can lead to unintended harmful outcomes, pharmacovigilance activities need to evolve to encompass novel complementary data streams, for example the biomedical literature available through MEDLINE. OBJECTIVES: (1) To review how the characteristics of MEDLINE indexing influence the identification of adverse drug events (ADEs); (2) to leverage this knowledge to inform the design of a system for extracting ADEs from MEDLINE indexing; and (3) to assess the specific contribution of some characteristics of MEDLINE indexing to the performance of this system. METHODS: We analyze the characteristics of MEDLINE indexing. We integrate three specific characteristics into the design of a system for extracting ADEs from MEDLINE indexing. We experimentally assess the specific contribution of these characteristics over a baseline system based on co-occurrence between drug descriptors qualified by adverse effects and disease descriptors qualified by chemically induced. RESULTS: Our system extracted 405,300 ADEs from 366,120 MEDLINE articles. The baseline system accounts for 297,093 ADEs (73%). 85,318 ADEs (21%) can be extracted only after integrating specific pre-coordinated MeSH descriptors and additional qualifiers. 22,889 ADEs (6%) can be extracted only after considering indirect links between the drug of interest and the descriptor that bears the ADE context. CONCLUSIONS: In this paper, we demonstrate significant improvement over a baseline approach to identifying ADEs from MEDLINE indexing, which mitigates some of the inherent limitations of MEDLINE indexing for pharmacovigilance. ADEs extracted from MEDLINE indexing are complementary to, not a replacement for, other sources.

A time-indexed reference standard of adverse drug reactions.

Undetected adverse drug reactions (ADRs) pose a major burden on the health system. Data mining methodologies designed to identify signals of novel ADRs are of deep importance for drug safety surveillance. The development and evaluation of these methodologies requires proper reference benchmarks. While progress has recently been made in developing such benchmarks, our understanding of the performance characteristics of the data mining methodologies is limited because existing benchmarks do not support prospective performance evaluations. We address this shortcoming by providing a reference standard to support prospective performance evaluations. The reference standard was systematically curated from drug labeling revisions, such as new warnings, which were issued and communicated by the US Food and Drug Administration in 2013. The reference standard includes 62 positive test cases and 75 negative controls, and covers 44 drugs and 38 events. We provide usage guidance and empirical support for the reference standard by applying it to analyze two data sources commonly mined for drug safety surveillance.

Dissimilar hepatotoxicity profiles of propylthiouracil and methimazole in children.

BACKGROUND: The antithyroid drugs propylthiouracil and methimazole were introduced for clinical use about 60 yr ago and are estimated to be used in more than 6000 children and adolescents per year in the United States. Over the years that these medications have been used, reports of adverse events involving hepatotoxicity have appeared. To date, there has not been a systematic and comparative evaluation of the adverse events associated with antithyroid drug use. OBJECTIVE: Our objective was to assess safety and hepatotoxicity profiles of propylthiouracil and methimazole by age in the U.S. Food and Drug Administration's Adverse Event Reporting System (AERS). DESIGN: We used the multi-item gamma-Poisson shrinker (MGPS) data mining algorithm to analyze more than 40 yr of safety data in AERS. MGPS uses a Bayesian model to calculate adjusted observed to expected ratios [empiric Bayes geometric mean (EBGM) values] for every drug-adverse event combination in AERS, focusing on hepatotoxicity events. RESULTS: MGPS identified higher-than-expected reporting of severe liver injury in pediatric patients treated with propylthiouracil but not with methimazole. Propylthiouracil had a high adjusted reporting ratio for severe liver injury (EBGM 17; 90% confidence interval = 11.5-24.1) in the group less than 17 yr old. The highest EBGM values for methimazole were with mild liver injury in the group 61 yr and older [EBGM 4.8 (3.3-6.8)], which consisted of cholestasis. Vasculitis was also observed for propylthiouracil in children and adolescents, reaching higher EBGM values than hepatotoxicity signals. CONCLUSIONS: MGPS detects higher-than-expected reporting of severe hepatotoxicity and vasculitis in children and adolescents with propylthiouracil but not with methimazole.

An evaluation of a data mining signal for amyotrophic lateral sclerosis and statins detected in FDA's spontaneous adverse event reporting system.

BACKGROUND: We detected disproportionate reporting of amyotrophic lateral sclerosis (ALS) with HMG-CoA-reductase inhibitors (statins) in the Food and Drug Administration's (FDA) spontaneous adverse event (AE) reporting system (AERS). PURPOSE: To describe the original ALS signal and to provide additional context for interpreting the signal by conducting retrospective analyses of data from long-term, placebo-controlled clinical trials of statins. METHODS: The ALS signal was detected using the multi-item gamma Poisson shrinker (MGPS) algorithm. All AERS cases of ALS reported in association with use of a statin were individually reviewed by two FDA neurologists. Manufacturers of lovastatin, pravastatin, simvastatin, fluvastatin, atorvastatin, cerivastatin, and rosuvastatin were requested to provide the number of cases of ALS diagnosed during all of their placebo-controlled statin trials that were at least 6 months in duration. RESULTS: There were 91 US and foreign reports of ALS with statins in AERS. The data mining signal scores for ALS and statins ranged from 8.5 to 1.6. Data were obtained from 41 statin clinical trials ranging in duration from 6 months to 5 years and representing approximately 200,000 patient-years of exposure to statin and approximately 200,000 patient-years of exposure to placebo. Nine cases of ALS were reported in statin-treated patients and 10 cases in placebo-treated patients. CONCLUSIONS: Although we observed a data mining signal for ALS with statins in FDA's AERS, retrospective analyses of 41 statin clinical trials did not reveal an increased incidence of ALS in subjects treated with a statin compared with placebo.

Atypical antipsychotics and pituitary tumors: a pharmacovigilance study.

STUDY OBJECTIVE: To analyze the disproportionality of reporting of hyperprolactinemia, galactorrhea, and pituitary tumors with seven widely used antipsychotic drugs. DESIGN: Retrospective pharmacovigilance study. DATA SOURCE: United States Food and Drug Administration's Adverse Event Reporting System (AERS) database. INTERVENTION: We initially identified higher-than-expected postmarketing reports of pituitary tumors associated with risperidone, a potent dopamine D2-receptor antagonist antipsychotic, by analyzing reporting patterns of these tumors in the AERS database. To further examine this association, we analyzed disproportionate reporting patterns of pituitary tumor reports for seven antipsychotics with different affinities for blocking D2 receptors: aripiprazole, clozapine, olanzapine, quetiapine, risperidone, ziprasidone, and haloperidol. MEASUREMENTS AND MAIN RESULTS: To conduct both of these analyses, we used the Multi-item Gamma Poisson Shrinker (MGPS) data mining algorithm applied to the AERS database. The MGPS uses a Bayesian model to calculate adjusted observed:expected ratios of drug-adverse event associations (Empiric Bayes Geometric Mean [EBGM] values) in huge drug safety databases. The higher the adjusted reporting ratio, or EBGM value, the greater the strength of the association between a drug and an adverse event. Risperidone had the highest adjusted reporting ratios for hyperprolactinemia (EBGM 34.9, 90% confidence interval [CI] 32.8-37.1]), galactorrhea (EBGM 19.9, 90% CI 18.6-21.4), and pituitary tumor (EBGM 18.7, 90% CI 14.9-23.3) among the seven antipsychotics, and one of the highest scores for all drugs in the AERS database. Some tumors were associated with visual field defects, hemorrhage, convulsions, surgery, and severe (>10-fold) prolactin elevations. The EBGM values for risperidone for these adverse events were higher in women, but high EBGM values for these events were also seen in men and children. Moreover, the rank order of the EBGM values for pituitary tumors corresponded to the affinities of these seven drugs for D2 receptors. CONCLUSION: Treatment with potent D2-receptor antagonists, such as risperidone, may be associated with pituitary tumors. These findings are consistent with animal (mice) studies and raise the need for clinical awareness and longitudinal studies.

Perspectives on the use of data mining in pharmaco-vigilance.

In the last 5 years, regulatory agencies and drug monitoring centres have been developing computerised data-mining methods to better identify reporting relationships in spontaneous reporting databases that could signal possible adverse drug reactions. At present, there are no guidelines or standards for the use of these methods in routine pharmaco-vigilance. In 2003, a group of statisticians, pharmaco-epidemiologists and pharmaco-vigilance professionals from the pharmaceutical industry and the US FDA formed the Pharmaceutical Research and Manufacturers of America-FDA Collaborative Working Group on Safety Evaluation Tools to review best practices for the use of these methods.In this paper, we provide an overview of: (i) the statistical and operational attributes of several currently used methods and their strengths and limitations; (ii) information about the characteristics of various postmarketing safety databases with which these tools can be deployed; (iii) analytical considerations for using safety data-mining methods and interpreting the results; and (iv) points to consider in integration of safety data mining with traditional pharmaco-vigilance methods. Perspectives from both the FDA and the industry are provided. Data mining is a potentially useful adjunct to traditional pharmaco-vigilance methods. The results of data mining should be viewed as hypothesis generating and should be evaluated in the context of other relevant data. The availability of a publicly accessible global safety database, which is updated on a frequent basis, would further enhance detection and communication about safety issues.

Pharmacovigilance in the 21st century: new systematic tools for an old problem.

The large number of adverse-event reports generated by marketed drugs and devices argues for the application of validated computerized algorithms to supplement traditional methods of detecting adverse-event signals. Difficulties in accurately estimating patient exposure and background rates for a given event in a specific population hinder risk estimation in spontaneous adverse-event databases. The United States Food and Drug Administration (FDA) is evaluating a Bayesian data mining system called Multi-item Gamma Poisson Shrinker (MGPS) to enhance the FDA's ability to monitor the safety of drugs, biologics, and vaccines after they have been approved for use. The MGPS computes adjusted higher-than-expected reporting relationships between drugs and adverse events across 35 years of data relative to internal background rates. The MGPS can also adjust for random noise by using a model derived from the data, and corrects for temporal trends and confounding related to age, sex, and other variables by stratifying over 900 categories. Signals can then be compared with or used in conjunction with other sources (e.g. clinical trials, general practice databases) to further study the adverse-event risk. The example of pancreatitis risk with atypical antipsychotics, valproic acid, and valproate is used to discuss the strengths and limitations of MGPS versus traditional methods. Validated data mining techniques offer great promise to enhance pharmacovigilance practices.

Use of screening algorithms and computer systems to efficiently signal higher-than-expected combinations of drugs and events in the US FDA's spontaneous reports database.

Since 1998, the US Food and Drug Administration (FDA) has been exploring new automated and rapid Bayesian data mining techniques. These techniques have been used to systematically screen the FDA's huge MedWatch database of voluntary reports of adverse drug events for possible events of concern. The data mining method currently being used is the Multi-Item Gamma Poisson Shrinker (MGPS) program that replaced the Gamma Poisson Shrinker (GPS) program we originally used with the legacy database. The MGPS algorithm, the technical aspects of which are summarised in this paper, computes signal scores for pairs, and for higher-order (e.g. triplet, quadruplet) combinations of drugs and events that are significantly more frequent than their pair-wise associations would predict. MGPS generates consistent, redundant, and replicable signals while minimising random patterns. Signals are generated without using external exposure data, adverse event background information, or medical information on adverse drug reactions. The MGPS interface streamlines multiple input-output processes that previously had been manually integrated. The system, however, cannot distinguish between already-known associations and new associations, so the reviewers must filter these events. In addition to detecting possible serious single-drug adverse event problems, MGPS is currently being evaluated to detect possible synergistic interactions between drugs (drug interactions) and adverse events (syndromes), and to detect differences among subgroups defined by gender and by age, such as paediatrics and geriatrics. In the current data, only 3.4% of all 1.2 million drug-event pairs ever reported (with frequencies > or = 1) generate signals [lower 95% confidence interval limit of the adjusted ratios of the observed counts over expected (O/E) counts (denoted EB05) of > or = 2]. The total frequency count that contributed to signals comprised 23% (2.4 million) of the total number, 10.4 million of drug-event pairs reported, greatly facilitating a more focused follow-up and evaluation. The algorithm provides an objective, systematic view of the data alerting reviewers to critically important, new safety signals. The study of signals detected by current methods, signals stored in the Center for Drug Evaluation and Research's Monitoring Adverse Reports Tracking System, and the signals regarding cerivastatin, a cholesterol-lowering drug voluntarily withdrawn from the market in August 2001, exemplify the potential of data mining to improve early signal detection. The operating characteristics of data mining in detecting early safety signals, exemplified by studying a drug recently well characterised by large clinical trials confirms our experience that the signals generated by data mining have high enough specificity to deserve further investigation. The application of these tools may ultimately improve usage recommendations.