Refining the impact of genetic evidence on clinical success.

The cost of drug discovery and development is driven primarily by failure1, with only about 10% of clinical programmes eventually receiving approval2-4. We previously estimated that human genetic evidence doubles the success rate from clinical development to approval5. In this study we leverage the growth in genetic evidence over the past decade to better understand the characteristics that distinguish clinical success and failure. We estimate the probability of success for drug mechanisms with genetic support is 2.6 times greater than those without. This relative success varies among therapy areas and development phases, and improves with increasing confidence in the causal gene, but is largely unaffected by genetic effect size, minor allele frequency or year of discovery. These results indicate we are far from reaching peak genetic insights to aid the discovery of targets for more effective drugs.

Semi-automation of keratopathy visual acuity grading of corneal events in belantamab mafodotin clinical trials: clinical decision support software.

Background: Belantamab mafodotin (belamaf) has demonstrated clinically meaningful antimyeloma activity in patients with heavily pretreated multiple myeloma. However, it is highly active against dividing cells, contributing to off-target adverse events, particularly ocular toxicity. Changes in best corrected visual acuity (BCVA) and corneal examination findings are routinely monitored to determine Keratopathy Visual Acuity (KVA) grade to inform belamaf dose modification. Objective: We aimed to develop a semiautomated mobile app to facilitate the grading of ocular events in clinical trials involving belamaf. Methods: The paper process was semiautomated by creating a library of finite-state automaton (FSA) models to represent all permutations of KVA grade changes from baseline BCVA readings. The transition states in the FSA models operated independently of eye measurement units (e.g., Snellen, logMAR, decimal) and provided a uniform approach to determining KVA grade changes. Together with the FSA, the complex decision tree for determining the grade change based on corneal examination findings was converted into logical statements for accurate and efficient overall KVA grade computation. First, a web-based user interface, conforming to clinical practice settings, was developed to simplify the input of key KVA grading criteria. Subsequently, a mobile app was developed that included additional guided steps to assist in clinical decision-making. Results: The app underwent a robust Good Clinical Practice validation process. Outcomes were reviewed by key stakeholders, our belamaf medical lead, and the systems integration team. The time to compute a patient's overall KVA grade using the Belamaf Eye Exam (BEE) app was reduced from a 20- to 30-min process to <1-2 min. The BEE app was well received, with most investigators surveyed selecting "satisfied" or "highly satisfied" for its accuracy and time efficiency. Conclusions: Our semiautomated approach provides for an accurate, simplified method of assessment of patients' corneal status that reduces errors and quickly delivers information critical for potential belamaf dose modifications. The app is currently available on the Apple iOS and Android platforms for use by investigators of the DREAMM clinical trials, and its use could easily be extended to the clinic to support healthcare providers who need to make informed belamaf treatment decisions.

Engaging Patients via Online Healthcare Fora: Three Pharmacovigilance Use Cases.

Increasingly, patient-generated safety insights are shared online, via general social media platforms or dedicated healthcare fora which give patients the opportunity to discuss their disease and treatment options. We evaluated three areas of potential interest for the use of social media in pharmacovigilance. To evaluate how social media may complement existing safety signal detection capabilities, we identified two use cases (drug/adverse event [AE] pairs) and then evaluated the frequency of AE discussions across a range of social media channels. Changes in frequency over time were noted in social media, then compared to frequency changes in Food and Drug Administration Adverse Event Reporting System (FAERS) data over the same time period using a traditional disproportionality method. Although both data sources showed increasing frequencies of AE discussions over time, the increase in frequency was greater in the FAERS data as compared to social media. To demonstrate the robustness of medical/AE insights of linked posts we manually reviewed 2,817 threads containing 21,313 individual posts from 3,601 unique authors. Posts from the same authors were linked together. We used a quality scoring algorithm to determine the groups of linked posts with the highest quality and manually evaluated the top 16 groups of posts. Most linked posts (12/16; 75%) contained all seven relevant medical insights assessed compared to only one (of 1,672) individual post. To test the capability of actively engage patients via social media to obtain follow-up AE information we identified and sent consents for follow-up to 39 individuals (through a third party). We sent target follow-up questions (identified by pharmacovigilance experts as critical for causality assessment) to those who consented. The number of people consenting to follow-up was low (20%), but receipt of follow-up was high (75%). We observed completeness of responses (37 out of 37 questions answered) and short average time required to receive the follow-up (1.8 days). Our findings indicate a limited use of social media data for safety signal detection. However, our research highlights two areas of potential value to pharmacovigilance: obtaining more complete medical/AE insights via longitudinal post linking and actively obtaining rapid follow-up information on AEs.

Identifying Barriers to Enrollment in Patient Pregnancy Registries: Building Evidence Through Crowdsourcing.

BACKGROUND: Enrollment in pregnancy registries is challenging despite substantial awareness-raising activities, generally resulting in low recruitment owing to limited safety data. Understanding patient and physician awareness of and attitudes toward pregnancy registries is needed to facilitate enrollment. Crowdsourcing, in which services, ideas, or content are obtained by soliciting contributions from a large group of people using web-based platforms, has shown promise for improving patient engagement and obtaining patient insights. OBJECTIVE: This study aimed to use web-based crowdsourcing platforms to evaluate Belimumab Pregnancy Registry (BPR) awareness among patients and physicians and to identify potential barriers to pregnancy registry enrollment with the BPR as a case study. METHODS: We conducted 2 surveys using separate web-based crowdsourcing platforms: Amazon Mechanical Turk (a 14-question patient survey) and Sermo RealTime (a 11-question rheumatologist survey). Eligible patients were women, aged 18-55 years; diagnosed with systemic lupus erythematosus (SLE); and pregnant, recently pregnant (within 2 years), or planning pregnancy. Eligible rheumatologists had prescribed belimumab and treated pregnant women. Responses were descriptively analyzed. RESULTS: Of 151 patient respondents over a 3-month period (n=88, 58.3% aged 26-35 years; n=149, 98.7% with mild or moderate SLE; and n=148, 98% from the United States), 51% (77/151) were currently or recently pregnant. Overall, 169 rheumatologists completed the survey within 48 hours, and 59.2% (100/169) were based in the United States. Belimumab exposure was reported by 41.7% (63/151) patients, whereas 51.7% (75/145) rheumatologists had prescribed belimumab to <5 patients, 25.5% (37/145) had prescribed to 5-10 patients, and 22.8% (33/145) had prescribed to >10 patients who were pregnant or trying to conceive. Of the patients exposed to belimumab, 51% (32/63) were BPR-aware, and 45.5% (77/169) of the rheumatologists were BPR-aware. Overall, 60% (38/63) of patients reported belimumab discontinuation because of pregnancy or planned pregnancy. Among the 77 BPR-aware rheumatologists, 70 (91%) referred patients to the registry. Concerns among rheumatologists who did not prescribe belimumab during pregnancy included unknown pregnancy safety profile (119/169, 70.4%), and 61.5% (104/169) reported their patients' concerns about the unknown pregnancy safety profile. Belimumab exposure during or recently after pregnancy or while trying to conceive was reported in patients with mild (6/64, 9%), moderate (22/85, 26%), or severe (1/2, 50%) SLE. Rheumatologists more commonly recommended belimumab for moderate (84/169, 49.7%) and severe (123/169, 72.8%) SLE than for mild SLE (36/169, 21.3%) for patients trying to conceive recently or currently pregnant. Overall, 81.6% (138/169) of the rheumatologists suggested a belimumab washout period before pregnancy of 0-30 days (44/138, 31.9%), 30-60 days (64/138, 46.4%), or >60 days (30/138, 21.7%). CONCLUSIONS: In this case, crowdsourcing efficiently obtained patient and rheumatologist input, with some patients with SLE continuing to use belimumab during or while planning a pregnancy. There was moderate awareness of the BPR among patients and physicians.

Artificial Intelligence Based on Machine Learning in Pharmacovigilance: A Scoping Review.

INTRODUCTION: Artificial intelligence based on machine learning has made large advancements in many fields of science and medicine but its impact on pharmacovigilance is yet unclear. OBJECTIVE: The present study conducted a scoping review of the use of artificial intelligence based on machine learning to understand how it is used for pharmacovigilance tasks, characterize differences with other fields, and identify opportunities to improve pharmacovigilance through the use of machine learning. DESIGN: The PubMed, Embase, Web of Science, and IEEE Xplore databases were searched to identify articles pertaining to the use of machine learning in pharmacovigilance published from the year 2000 to September 2021. After manual screening of 7744 abstracts, a total of 393 papers met the inclusion criteria for further analysis. Extraction of key data on study design, data sources, sample size, and machine learning methodology was performed. Studies with the characteristics of good machine learning practice were defined and manual review focused on identifying studies that fulfilled these criteria and results that showed promise. RESULTS: The majority of studies (53%) were focused on detecting safety signals using traditional statistical methods. Of the studies that used more recent machine learning methods, 61% used off-the-shelf techniques with minor modifications. Temporal analysis revealed that newer methods such as deep learning have shown increased use in recent years. We found only 42 studies (10%) that reflect current best practices and trends in machine learning. In the subset of 154 papers that focused on data intake and ingestion, 30 (19%) were found to incorporate the same best practices. CONCLUSION: Advances from artificial intelligence have yet to fully penetrate pharmacovigilance, although recent studies show signs that this may be changing.

Developing Crowdsourced Training Data Sets for Pharmacovigilance Intelligent Automation.

INTRODUCTION: Machine learning offers an alluring solution to developing automated approaches to the increasing individual case safety report burden being placed upon pharmacovigilance. Leveraging crowdsourcing to annotate unstructured data may provide accurate, efficient, and contemporaneous training data sets in support of machine learning. OBJECTIVE: The objective of this study was to evaluate whether crowdsourcing can be used to accurately and efficiently develop training data sets in support of pharmacovigilance automation. MATERIALS AND METHODS: Pharmacovigilance experts created a reference dataset by reviewing 15,490 de-identified social media posts of narratives pertaining to 15 drugs and 22 medically relevant topics. A random sampling of posts from the reference dataset was published on Amazon Turk and its users (Turkers) were asked a series of questions about those same medical concepts. Accuracy, price elasticity, and time efficiency were evaluated. RESULTS: Accuracy of crowdsourced curation exceeded 90% when compared to the reference dataset and was completed in about 5% of the time. There was an increase in time efficiency with higher pay, but there was no significant difference in accuracy. Additionally, having a social media post reviewed by more than one Turker (using a voting system) did not offer significant improvements in terms of accuracy. CONCLUSIONS: Crowdsourcing is an accurate and efficient method that can be used to develop training data sets in support of pharmacovigilance automation. More research is needed to better understand the breadth and depth of possible uses as well as strengths, limitations, and generalizability of results.

Using Social Listening Data to Monitor Misuse and Nonmedical Use of Bupropion: A Content Analysis.

BACKGROUND: The nonmedical use of pharmaceutical products has become a significant public health concern. Traditionally, the evaluation of nonmedical use has focused on controlled substances with addiction risk. Currently, there is no effective means of evaluating the nonmedical use of noncontrolled antidepressants. OBJECTIVE: Social listening, in the context of public health sometimes called infodemiology or infoveillance, is the process of identifying and assessing what is being said about a company, product, brand, or individual, within forms of electronic interactive media. The objectives of this study were (1) to determine whether content analysis of social listening data could be utilized to identify posts discussing potential misuse or nonmedical use of bupropion and two comparators, amitriptyline and venlafaxine, and (2) to describe and characterize these posts. METHODS: Social listening was performed on all publicly available posts cumulative through July 29, 2015, from two harm-reduction Web forums, Bluelight and Opiophile, which mentioned the study drugs. The acquired data were stripped of personally identifiable identification (PII). A set of generic, brand, and vernacular product names was used to identify product references in posts. Posts were obtained using natural language processing tools to identify vernacular references to drug misuse-related Preferred Terms from the English Medical Dictionary for Regulatory Activities (MedDRA) version 18 terminology. Posts were reviewed manually by coders, who extracted relevant details. RESULTS: A total of 7756 references to at least one of the study antidepressants were identified within posts gathered for this study. Of these posts, 668 (8.61%, 668/7756) referenced misuse or nonmedical use of the drug, with bupropion accounting for 438 (65.6%, 438/668). Of the 668 posts, nonmedical use was discouraged by 40.6% (178/438), 22% (22/100), and 18.5% (24/130) and encouraged by 12.3% (54/438), 10% (10/100), and 10.8% (14/130) for bupropion, amitriptyline, and venlafaxine, respectively. The most commonly reported desired effects were similar to stimulants with bupropion, sedatives with amitriptyline, and dissociatives with venlafaxine. The nasal route of administration was most frequently reported for bupropion, whereas the oral route was most frequently reported for amitriptyline and venlafaxine. Bupropion and venlafaxine were most commonly procured from health care providers, whereas amitriptyline was most commonly obtained or stolen from a third party. The Fleiss kappa for interrater agreement among 20 items with 7 categorical response options evaluated by all 11 raters was 0.448 (95% CI 0.421-0.457). CONCLUSIONS: Social listening, conducted in collaboration with harm-reduction Web forums, offers a valuable new data source that can be used for monitoring nonmedical use of antidepressants. Additional work on the capabilities of social listening will help further delineate the benefits and limitations of this rapidly evolving data source.

Performance of Stratified and Subgrouped Disproportionality Analyses in Spontaneous Databases.

INTRODUCTION: Disproportionality analyses are used in many organisations to identify adverse drug reactions (ADRs) from spontaneous report data. Reporting patterns vary over time, with patient demographics, and between different geographical regions, and therefore subgroup analyses or adjustment by stratification may be beneficial. OBJECTIVE: The objective of this study was to evaluate the performance of subgroup and stratified disproportionality analyses for a number of key covariates within spontaneous report databases of differing sizes and characteristics. METHODS: Using a reference set of established ADRs, signal detection performance (sensitivity and precision) was compared for stratified, subgroup and crude (unadjusted) analyses within five spontaneous report databases (two company, one national and two international databases). Analyses were repeated for a range of covariates: age, sex, country/region of origin, calendar time period, event seriousness, vaccine/non-vaccine, reporter qualification and report source. RESULTS: Subgroup analyses consistently performed better than stratified analyses in all databases. Subgroup analyses also showed benefits in both sensitivity and precision over crude analyses for the larger international databases, whilst for the smaller databases a gain in precision tended to result in some loss of sensitivity. Additionally, stratified analyses did not increase sensitivity or precision beyond that associated with analytical artefacts of the analysis. The most promising subgroup covariates were age and region/country of origin, although this varied between databases. CONCLUSIONS: Subgroup analyses perform better than stratified analyses and should be considered over the latter in routine first-pass signal detection. Subgroup analyses are also clearly beneficial over crude analyses for larger databases, but further validation is required for smaller databases.

Social Media Listening for Routine Post-Marketing Safety Surveillance.

INTRODUCTION: Post-marketing safety surveillance primarily relies on data from spontaneous adverse event reports, medical literature, and observational databases. Limitations of these data sources include potential under-reporting, lack of geographic diversity, and time lag between event occurrence and discovery. There is growing interest in exploring the use of social media ('social listening') to supplement established approaches for pharmacovigilance. Although social listening is commonly used for commercial purposes, there are only anecdotal reports of its use in pharmacovigilance. Health information posted online by patients is often publicly available, representing an untapped source of post-marketing safety data that could supplement data from existing sources. OBJECTIVES: The objective of this paper is to describe one methodology that could help unlock the potential of social media for safety surveillance. METHODS: A third-party vendor acquired 24 months of publicly available Facebook and Twitter data, then processed the data by standardizing drug names and vernacular symptoms, removing duplicates and noise, masking personally identifiable information, and adding supplemental data to facilitate the review process. The resulting dataset was analyzed for safety and benefit information. RESULTS: In Twitter, a total of 6,441,679 Medical Dictionary for Regulatory Activities (MedDRA(®)) Preferred Terms (PTs) representing 702 individual PTs were discussed in the same post as a drug compared with 15,650,108 total PTs representing 946 individual PTs in Facebook. Further analysis revealed that 26 % of posts also contained benefit information. CONCLUSION: Social media listening is an important tool to augment post-marketing safety surveillance. Much work remains to determine best practices for using this rapidly evolving data source.

The support of human genetic evidence for approved drug indications.

Over a quarter of drugs that enter clinical development fail because they are ineffective. Growing insight into genes that influence human disease may affect how drug targets and indications are selected. However, there is little guidance about how much weight should be given to genetic evidence in making these key decisions. To answer this question, we investigated how well the current archive of genetic evidence predicts drug mechanisms. We found that, among well-studied indications, the proportion of drug mechanisms with direct genetic support increases significantly across the drug development pipeline, from 2.0% at the preclinical stage to 8.2% among mechanisms for approved drugs, and varies dramatically among disease areas. We estimate that selecting genetically supported targets could double the success rate in clinical development. Therefore, using the growing wealth of human genetic data to select the best targets and indications should have a measurable impact on the successful development of new drugs.

A proposed modification to Hy's law and Edish criteria in oncology clinical trials using aggregated historical data.

PURPOSE: Identifying drug-induced liver injury is a critical task in drug development and postapproval real-world care. Severe liver injury is identified by the liver chemistry threshold of alanine aminotransferase (ALT) >3× upper limit of normal (ULN) and bilirubin >2× ULN, termed Hy's law by the Food and Drug Administration. These thresholds require discontinuation of the causative drug and are seldom exceeded in most patient populations. However, because maintenance of therapy is critical in the treatment of advanced cancer, customized thresholds may be useful in oncology patient populations, particularly for those with baseline liver chemistries elevations. METHODS: Liver chemistry data from 31 aggregated oncology clinical trials were modeled through a truncated robust multivariate outlier detection (TRMOD) method to develop the decision boundary or threshold for examining liver injury in oncology clinical trials. RESULTS: The boundary of TRMOD identified outliers with an ALT limit 5.0× ULN and total bilirubin limit 2.7× ULN. In addition, TRMOD was applied to the aggregated oncology data to examine fold-baseline ALT and total bilirubin, revealing limits of ALT 6.9× baseline and bilirubin 6.5× baseline. Similar ALT and bilirubin threshold limits were observed for oncology patients both with and without liver metastases. CONCLUSIONS: These higher liver chemistry thresholds examining fold-ULN and fold-baseline data may be valuable in identifying potential severe liver injury and detecting liver safety signals of clinical concern in oncology clinical trials and postapproval settings while helping to avoid premature discontinuation of curative therapy.

Predicting breast cancer chemotherapeutic response using a novel tool for microarray data analysis.

We developed a novel tool for microarray data analysis that can parsimoniously discover highly predictive genes by finding the optimal trade off between fold change and t-test p value through rigorous cross validation. In addition to find a small set of highly predictive genes, the tool also has a procedure that recursively discovers and removes predictive genes from the dataset until no such genes can be found. We applied our tool to a public breast cancer dataset with the goal to discover genes that can predict patient’s response to a preoperative chemotherapy. The results show that estrogen receptor (ER) gene is the most important gene to predict chemotherapeutic response and no gene signatures can add much clinical benefit for the whole patient population. We further identified a clinically homogenous subgroup of patients (ER-negative, PR-negative and HER2-negative) whose response to the chemotherapy can be reasonably predicted. Many of the discovered predictive markers for this subgroup of patients were successfully validated using a blinded validation set.

Validation of multivariate outlier detection analyses used to identify potential drug-induced liver injury in clinical trial populations.

BACKGROUND: Potential severe liver injury is identified in clinical trials by ALT >3 × upper limits of normal (ULN) and total bilirubin >2 × ULN, and termed 'Hy's Law' by the US FDA. However, there is limited evidence or validation of these thresholds in clinical trial populations. Using liver chemistry data from clinical trials, decision boundaries were built empirically with truncated robust multivariate outlier detection (TRMOD), in a statistically robust manner, and then compared with these fixed thresholds. Additionally, as the analysis of liver chemistry change from baseline has been recently suggested for the identification of liver signals, fold-baseline data was also assessed. OBJECTIVE: The aim of the study was to examine and validate the performance of fixed and empirically derived thresholds for severe liver injury in generally healthy clinical trial populations (i.e. populations without underlying renal, haematological or liver disease). METHODS: Using phase II-IV clinical trial data, ALT and total bilirubin data were analysed using outlier detection methods to compare with empirically derived and fixed thresholds of the FDA's Hy's Law limits, which were then assessed graphically with the FDA's evaluation of Drug-Induced Serious Hepatotoxicity (eDISH) assessing fold-ULN, as well as a modified eDISH (mDISH) to assess fold-baseline liver chemistries. Data from 28 phase II-IV clinical trials conducted by GlaxoSmithKline were aggregated and analysed by the TRMOD algorithm to create decision boundaries. The data consisted of 18 672 predominantly female subjects with a mean age of 44 years and without known liver disease. RESULTS: Among generally healthy clinical trial subjects, the empirically-derived TRMOD boundaries were approximately equivalent to 'Hy's Law'. TRMOD boundaries for identifying outliers were an ALT limit of 3.4 × ULN and a bilirubin limit of 2.1 × ULN, compared with the FDA's 'Hy's Law' of 3 × ULN and bilirubin 2 × ULN. Inter-laboratory data variations were observed across the 28 studies, and were diminished by use of baseline-corrected data. By applying TRMOD to baseline-corrected data, these boundaries became ALT limit of 3.8 × baseline and bilirubin limit of 4.8 × baseline. Cumulative incidence plots of liver signals identified over time were examined. TRMOD analyses identified normative boundaries and outliers that provide comparative data to detect liver signals in similar trial populations. CONCLUSIONS: TRMOD liver chemistry analyses of clinical trial data in generally healthy subjects have confirmed the FDA's Hy's Law threshold as a robust means of detecting liver safety outliers. TRMOD evaluation of liver chemistry data, by both fold-ULN and fold-baseline, provides complementary analyses and valuable normative data for comparison in similar patient populations. No liver signal is present when new clinical trial data from similar patient populations lies within these normative boundaries. Use of baseline-corrected data diminishes inter-laboratory variation and may be more sensitive to possible drug effects. We suggest examining liver chemistries using graphical depictions of both ULN-corrected data (eDISH) and baseline-corrected data (mDISH), as complementary methods.

Development and evaluation of a common data model enabling active drug safety surveillance using disparate healthcare databases.

OBJECTIVE: Active drug safety surveillance may be enhanced by analysis of multiple observational healthcare databases, including administrative claims and electronic health records. The objective of this study was to develop and evaluate a common data model (CDM) enabling rapid, comparable, systematic analyses across disparate observational data sources to identify and evaluate the effects of medicines. DESIGN: The CDM uses a person-centric design, with attributes for demographics, drug exposures, and condition occurrence. Drug eras, constructed to represent periods of persistent drug use, are derived from available elements from pharmacy dispensings, prescriptions written, and other medication history. Condition eras aggregate diagnoses that occur within a single episode of care. Drugs and conditions from source data are mapped to biomedical ontologies to standardize terminologies and enable analyses of higher-order effects. MEASUREMENTS: The CDM was applied to two source types: an administrative claims and an electronic medical record database. Descriptive statistics were used to evaluate transformation rules. Two case studies demonstrate the ability of the CDM to enable standard analyses across disparate sources: analyses of persons exposed to rofecoxib and persons with an acute myocardial infarction. RESULTS: Over 43 million persons, with nearly 1 billion drug exposures and 3.7 billion condition occurrences from both databases were successfully transformed into the CDM. An analysis routine applied to transformed data from each database produced consistent, comparable results. CONCLUSION: A CDM can normalize the structure and content of disparate observational data, enabling standardized analyses that are meaningfully comparable when assessing the effects of medicines.

Toward automating an inference model on unstructured terminologies: OXMIS case study.

Most modern biomedical vocabularies employ some hierarchical representation that provides a "broader/narrower" meaning relationship among the "codes" or "concepts" found within them. Often, however, we may find within the clinical setting the creation and curation of unstructured custom vocabularies used in the everyday practice of classifying and categorizing clinical data and findings.A significant and widely used example of this lies in the General Practice Research Database which makes use of the Oxford Medical Information Systems (OXMIS) coding scheme to represent drugs and medical conditions. This scheme is intrinsically unstructured, is generally regarded as disorganized, and is not amenable to comparison with other hierarchically structured medical coding schemes. To improve processes of data analysis and extraction, we define a semantically meaningful representation of the OXMIS codes by way of the Unified Medical Language System (UMLS) Metathesaurus. A structure-imposing ontology mapping is created, and this process provides a complete illustration of a general semantic mapping technique applicable to unstructured biomedical terminologies.