Validation of a Major and Clinically Relevant Nonmajor Bleeding Phenotyping Algorithm on Electronic Health Records.

PURPOSE: Bleeding is an important health outcome of interest in epidemiological studies. We aimed to develop and validate rule-based algorithms to identify (1) major bleeding and (2) all clinically relevant bleeding (CRB) (composite of major and all clinically relevant nonmajor bleeding) within real-world electronic healthcare data. METHODS: We took a random sample (n = 1630) of inpatient admissions to Singapore public healthcare institutions in 2019 and 2020, stratifying by hospital and year. We included patients of all age groups, sex, and ethnicities. Presence of major bleeding and CRB were ascertained by two annotators through chart review. A total of 630 and 1000 records were used for algorithm development and validation, respectively. We formulated two algorithms: sensitivity- and positive predictive value (PPV)-optimized algorithms. A combination of hemoglobin test patterns and diagnosis codes were used in the final algorithms. RESULTS: During validation, diagnosis codes alone yielded low sensitivities for major bleeding (0.16) and CRB (0.24), although specificities and PPV were high (>0.97). For major bleeding, the sensitivity-optimized algorithm had much higher sensitivity and negative predictive values (NPVs) (sensitivity = 0.94, NPV = 1.00), however false positive rates were also relatively high (specificity = 0.90, PPV = 0.34). PPV-optimized algorithm had improved specificity and PPV (specificity = 0.96, PPV = 0.52), with little reduction in sensitivity and NPV (sensitivity = 0.88, NPV = 0.99). For CRB events, our algorithms had lower sensitivities (0.50-0.56). CONCLUSIONS: The use of diagnosis codes alone misses many genuine major bleeding events. We have developed major bleeding algorithms with high sensitivities, which can ascertain events within populations of interest.

Combining Machine Learning with a Rule-Based Algorithm to Detect and Identify Related Entities of Documented Adverse Drug Reactions on Hospital Discharge Summaries.

INTRODUCTION: Discharge summaries contain valuable information about adverse drug reactions, but their unstructured nature makes them challenging to analyse and use as a signal source for pharmacovigilance. Machine learning has shown promise in identifying discharge summaries that contain related drug-adverse event pairs but has fared relatively poorer in entity extraction. METHODS: A hybrid model is developed combining rule-based and machine learning algorithms using discharge summaries with the aim of maximising capture of related drug-adverse event pairs. The rule first identifies segments containing adverse event entities within a 100-character distance from a drug term; machine learning subsequently estimates the relatedness of the drug and adverse event entities contained. The approach is validated on four independent datasets that are temporally and geographically separated from model development data. The impact of restricted drug-adverse event pair detection on recall is evaluated by using two of the four validation datasets that do not impose rule-based restrictions to annotations. RESULTS: The hybrid model achieves a recall of 0.80 (fivefold cross validation), 0.80 (temporal) and 0.76 (geographical) on validation using datasets containing only pre-identified target text segments that fulfil the rule-based algorithm criteria. When tested on datasets that additionally contained drug-adverse event pairs not restricted by the rule-based criteria, recall of the model declines to 0.68 and 0.62 on temporally and geographically separated datasets, respectively. CONCLUSIONS: The proposed hybrid model demonstrates reasonable generalisability on external validation. Rule-based restriction of the detection space results in an approximately 12-14% reduction in recall but improves identification of the related drug and adverse event terms.

Quantitative data mining in signal detection: the Singapore experience.

Background: In Singapore, the Health Sciences Authority (HSA) reviews an average of 20,000 spontaneous adverse event (AE) reports yearly. Potential safety signals are identified manually and discussed on a weekly basis. In this study, we compared the use of four quantitative data mining (QDM) methods with weekly manual review to determine if signals of disproportionate reporting (SDRs) can improve the efficiency of manual reviews and thereby enhance drug safety signal detection.Methods: We formulated a QDM triage strategy to reduce the number of SDRs for weekly review and compared the results against those derived from manual reviews alone for the same 6-month period. We then incorporated QDM triage into the manual review workflow for the subsequent two 6-month periods and made further comparisons against QDM triage alone.Results: The incorporation of QDM triage into routine manual reviews resulted in a reduction of 20% to 30% in the number of drug-AE pairs identified for further evaluation. Sequential Probability Ratio Test (SPRT) detected more signals that mirror human manual signal detection than the other three methods.Conclusions: The adoption of QDM triage into our manual reviews is a more efficient way forward in signal detection, avoiding missing important drug safety signals.

Detecting adverse drug reactions in discharge summaries of electronic medical records using Readpeer.

BACKGROUND: Hospital discharge summaries offer a potentially rich resource to enhance pharmacovigilance efforts to evaluate drug safety in real-world clinical practice. However, it is infeasible for experts to read through all discharge summaries to find cases of drug-adverse event (AE) relations. PURPOSE: The objective of this paper is to develop a natural language processing (NLP) framework to detect drug-AE relations from unstructured hospital discharge summaries. BASIC PROCEDURES: An NLP algorithm was designed using customized dictionaries of drugs, adverse event (AE) terms, and rules based on trigger phrases, negations, fuzzy logic and word distances to recognize drug, AE terms and to detect drug-AE relations. Furthermore, a customized annotation tool was developed to facilitate expert review of discharge summaries from a tertiary hospital in Singapore in 2011. MAIN FINDINGS: A total of 33 trial sets with 50 to 100 records per set were evaluated (1620 discharge summaries) by our algorithm and reviewed by pharmacovigilance experts. After every 6 trial sets, drug and AE dictionaries were updated, and rules were modified to improve the system. Excellent performance was achieved for drug and AE entity recognition with over 92% precision and recall. On the final 6 sets of discharge summaries (600 records), our algorithm achieved 75% precision and 59% recall for identification of valid drug-AE relations. PRINCIPAL CONCLUSIONS: Adverse drug reactions are a significant contributor to health care costs and utilization. Our algorithm is not restricted to particular drugs, drug classes or specific medical specialties, which is an important attribute for a national regulatory authority to carry out comprehensive safety monitoring of drug products. Drug and AE dictionaries may be updated periodically to ensure that the tool remains relevant for performing surveillance activities. The development of the algorithm, and the ease of reviewing and correcting the results of the algorithm as part of an iterative machine learning process, is an important step towards use of hospital discharge summaries for an active pharmacovigilance program.

Retrospective Study of Reported Adverse Events Due to Complementary Health Products in Singapore From 2010 to 2016.

The objective of this study is to collate and analyse adverse event reports associated with the use of complementary health products (CHP) submitted to the Health Sciences Authority (HSA) of Singapore for the period 2010-2016 to identify various trends and signals for pharmacovigilance purposes. A total of 147,215 adverse event reports suspected to be associated with pharmaceutical products and CHP were received by HSA between 2010 and 2016. Of these, 143,191 (97.3%) were associated with chemical drugs, 1,807 (1.2%) with vaccines, 1,324 (0.9%) with biological drugs (biologics), and 893 (0.6%) with CHP. The number of adverse event reports associated with Chinese Proprietary Medicine, other complementary medicine and health supplements are presented. Eight hundred and ninety three adverse event reports associated with CHP in the 7-year period have been successfully collated and analyzed. In agreement with other studies, adverse events related to the "skin and appendages disorders" were the most commonly reported. Most of the cases involved dermal allergies (e.g., rashes) associated with the use of glucosamine products and most of the adulterated products were associated with the illegal addition of undeclared drugs for pain relief. Dexamethasone, chlorpheniramine, and piroxicam were the most common adulterants detected. Reporting suspected adverse events is strongly encouraged even if the causality is not confirmed because any signs of clustering will allow rapid regulatory actions to be taken. The findings from this study help to create greater awareness on the health risks, albeit low, when consuming CHP and dispelling the common misconception that "natural" means "safe." In particular, healthcare professionals and the general public should be aware of potential adulteration of CHP. The analysis of spontaneously reported adverse events is an important surveillance system in monitoring the safety of CHP and helps in the understanding of the risk associated with the use of such products. Greater collaboration and communication between healthcare professionals, regulators, patients, manufacturers, researchers, and the general public are important to ensure the quality and safety of CHP.

Drug-induced liver injury associated with Complementary and Alternative Medicine: a review of adverse event reports in an Asian community from 2009 to 2014.

BACKGROUND: The use of Complementary and Alternative Medicine (CAM) has been increasing over the years. A recent review of adverse event reports (AERs) associated with CAM in Singapore found a notable number of AERs submitted. The objectives of this study are to analyse hepatotoxicity cases associated with CAM in Singapore based on spontaneous adverse event reporting to the Health Sciences Authority (HSA), and to highlight safety signals for specific herbal ingredients. METHODS: AERs associated with CAM and hepatotoxicity submitted to the Vigilance and Compliance Branch (VCB) of the HSA from 2009 to 2014 were compiled. The following information was extracted and analysed: Demographic information; time to onset; hospitalisation status; outcome; type of hepatotoxicity; ingredients of CAM, and the total daily doses (TDD); concurrent western medicines and health supplements; and reporter details. RESULTS: Fifty-seven reports were eligible for analysis. Thirty-five (61.4 %) cases involved Traditional Chinese Medicine (TCM). The Roussel Uclaf Causality Assessment Method was applied in 29 (82.9 %) of these cases, and the median score was 4 (range: 1-8). Chai Hu (Radix bupleuri) was suspected in 11 (31.4 %) cases. TDDs of most ingredients were within recommended doses of the Chinese Pharmacopoeia. CONCLUSIONS: Drug-induced liver injury is still poorly understood and more objective assessments are warranted. Reporting of adverse events should be strongly advocated to facilitate future analyses and the understanding of risk-benefit profiles of CAM.