Refining the impact of genetic evidence on clinical success.

The cost of drug discovery and development is driven primarily by failure1, with only about 10% of clinical programmes eventually receiving approval2-4. We previously estimated that human genetic evidence doubles the success rate from clinical development to approval5. In this study we leverage the growth in genetic evidence over the past decade to better understand the characteristics that distinguish clinical success and failure. We estimate the probability of success for drug mechanisms with genetic support is 2.6 times greater than those without. This relative success varies among therapy areas and development phases, and improves with increasing confidence in the causal gene, but is largely unaffected by genetic effect size, minor allele frequency or year of discovery. These results indicate we are far from reaching peak genetic insights to aid the discovery of targets for more effective drugs.

Semi-automation of keratopathy visual acuity grading of corneal events in belantamab mafodotin clinical trials: clinical decision support software.

Background: Belantamab mafodotin (belamaf) has demonstrated clinically meaningful antimyeloma activity in patients with heavily pretreated multiple myeloma. However, it is highly active against dividing cells, contributing to off-target adverse events, particularly ocular toxicity. Changes in best corrected visual acuity (BCVA) and corneal examination findings are routinely monitored to determine Keratopathy Visual Acuity (KVA) grade to inform belamaf dose modification. Objective: We aimed to develop a semiautomated mobile app to facilitate the grading of ocular events in clinical trials involving belamaf. Methods: The paper process was semiautomated by creating a library of finite-state automaton (FSA) models to represent all permutations of KVA grade changes from baseline BCVA readings. The transition states in the FSA models operated independently of eye measurement units (e.g., Snellen, logMAR, decimal) and provided a uniform approach to determining KVA grade changes. Together with the FSA, the complex decision tree for determining the grade change based on corneal examination findings was converted into logical statements for accurate and efficient overall KVA grade computation. First, a web-based user interface, conforming to clinical practice settings, was developed to simplify the input of key KVA grading criteria. Subsequently, a mobile app was developed that included additional guided steps to assist in clinical decision-making. Results: The app underwent a robust Good Clinical Practice validation process. Outcomes were reviewed by key stakeholders, our belamaf medical lead, and the systems integration team. The time to compute a patient's overall KVA grade using the Belamaf Eye Exam (BEE) app was reduced from a 20- to 30-min process to <1-2 min. The BEE app was well received, with most investigators surveyed selecting "satisfied" or "highly satisfied" for its accuracy and time efficiency. Conclusions: Our semiautomated approach provides for an accurate, simplified method of assessment of patients' corneal status that reduces errors and quickly delivers information critical for potential belamaf dose modifications. The app is currently available on the Apple iOS and Android platforms for use by investigators of the DREAMM clinical trials, and its use could easily be extended to the clinic to support healthcare providers who need to make informed belamaf treatment decisions.

Engaging Patients via Online Healthcare Fora: Three Pharmacovigilance Use Cases.

Increasingly, patient-generated safety insights are shared online, via general social media platforms or dedicated healthcare fora which give patients the opportunity to discuss their disease and treatment options. We evaluated three areas of potential interest for the use of social media in pharmacovigilance. To evaluate how social media may complement existing safety signal detection capabilities, we identified two use cases (drug/adverse event [AE] pairs) and then evaluated the frequency of AE discussions across a range of social media channels. Changes in frequency over time were noted in social media, then compared to frequency changes in Food and Drug Administration Adverse Event Reporting System (FAERS) data over the same time period using a traditional disproportionality method. Although both data sources showed increasing frequencies of AE discussions over time, the increase in frequency was greater in the FAERS data as compared to social media. To demonstrate the robustness of medical/AE insights of linked posts we manually reviewed 2,817 threads containing 21,313 individual posts from 3,601 unique authors. Posts from the same authors were linked together. We used a quality scoring algorithm to determine the groups of linked posts with the highest quality and manually evaluated the top 16 groups of posts. Most linked posts (12/16; 75%) contained all seven relevant medical insights assessed compared to only one (of 1,672) individual post. To test the capability of actively engage patients via social media to obtain follow-up AE information we identified and sent consents for follow-up to 39 individuals (through a third party). We sent target follow-up questions (identified by pharmacovigilance experts as critical for causality assessment) to those who consented. The number of people consenting to follow-up was low (20%), but receipt of follow-up was high (75%). We observed completeness of responses (37 out of 37 questions answered) and short average time required to receive the follow-up (1.8 days). Our findings indicate a limited use of social media data for safety signal detection. However, our research highlights two areas of potential value to pharmacovigilance: obtaining more complete medical/AE insights via longitudinal post linking and actively obtaining rapid follow-up information on AEs.

Identifying Barriers to Enrollment in Patient Pregnancy Registries: Building Evidence Through Crowdsourcing.

BACKGROUND: Enrollment in pregnancy registries is challenging despite substantial awareness-raising activities, generally resulting in low recruitment owing to limited safety data. Understanding patient and physician awareness of and attitudes toward pregnancy registries is needed to facilitate enrollment. Crowdsourcing, in which services, ideas, or content are obtained by soliciting contributions from a large group of people using web-based platforms, has shown promise for improving patient engagement and obtaining patient insights. OBJECTIVE: This study aimed to use web-based crowdsourcing platforms to evaluate Belimumab Pregnancy Registry (BPR) awareness among patients and physicians and to identify potential barriers to pregnancy registry enrollment with the BPR as a case study. METHODS: We conducted 2 surveys using separate web-based crowdsourcing platforms: Amazon Mechanical Turk (a 14-question patient survey) and Sermo RealTime (a 11-question rheumatologist survey). Eligible patients were women, aged 18-55 years; diagnosed with systemic lupus erythematosus (SLE); and pregnant, recently pregnant (within 2 years), or planning pregnancy. Eligible rheumatologists had prescribed belimumab and treated pregnant women. Responses were descriptively analyzed. RESULTS: Of 151 patient respondents over a 3-month period (n=88, 58.3% aged 26-35 years; n=149, 98.7% with mild or moderate SLE; and n=148, 98% from the United States), 51% (77/151) were currently or recently pregnant. Overall, 169 rheumatologists completed the survey within 48 hours, and 59.2% (100/169) were based in the United States. Belimumab exposure was reported by 41.7% (63/151) patients, whereas 51.7% (75/145) rheumatologists had prescribed belimumab to <5 patients, 25.5% (37/145) had prescribed to 5-10 patients, and 22.8% (33/145) had prescribed to >10 patients who were pregnant or trying to conceive. Of the patients exposed to belimumab, 51% (32/63) were BPR-aware, and 45.5% (77/169) of the rheumatologists were BPR-aware. Overall, 60% (38/63) of patients reported belimumab discontinuation because of pregnancy or planned pregnancy. Among the 77 BPR-aware rheumatologists, 70 (91%) referred patients to the registry. Concerns among rheumatologists who did not prescribe belimumab during pregnancy included unknown pregnancy safety profile (119/169, 70.4%), and 61.5% (104/169) reported their patients' concerns about the unknown pregnancy safety profile. Belimumab exposure during or recently after pregnancy or while trying to conceive was reported in patients with mild (6/64, 9%), moderate (22/85, 26%), or severe (1/2, 50%) SLE. Rheumatologists more commonly recommended belimumab for moderate (84/169, 49.7%) and severe (123/169, 72.8%) SLE than for mild SLE (36/169, 21.3%) for patients trying to conceive recently or currently pregnant. Overall, 81.6% (138/169) of the rheumatologists suggested a belimumab washout period before pregnancy of 0-30 days (44/138, 31.9%), 30-60 days (64/138, 46.4%), or >60 days (30/138, 21.7%). CONCLUSIONS: In this case, crowdsourcing efficiently obtained patient and rheumatologist input, with some patients with SLE continuing to use belimumab during or while planning a pregnancy. There was moderate awareness of the BPR among patients and physicians.

Artificial Intelligence Based on Machine Learning in Pharmacovigilance: A Scoping Review.

INTRODUCTION: Artificial intelligence based on machine learning has made large advancements in many fields of science and medicine but its impact on pharmacovigilance is yet unclear. OBJECTIVE: The present study conducted a scoping review of the use of artificial intelligence based on machine learning to understand how it is used for pharmacovigilance tasks, characterize differences with other fields, and identify opportunities to improve pharmacovigilance through the use of machine learning. DESIGN: The PubMed, Embase, Web of Science, and IEEE Xplore databases were searched to identify articles pertaining to the use of machine learning in pharmacovigilance published from the year 2000 to September 2021. After manual screening of 7744 abstracts, a total of 393 papers met the inclusion criteria for further analysis. Extraction of key data on study design, data sources, sample size, and machine learning methodology was performed. Studies with the characteristics of good machine learning practice were defined and manual review focused on identifying studies that fulfilled these criteria and results that showed promise. RESULTS: The majority of studies (53%) were focused on detecting safety signals using traditional statistical methods. Of the studies that used more recent machine learning methods, 61% used off-the-shelf techniques with minor modifications. Temporal analysis revealed that newer methods such as deep learning have shown increased use in recent years. We found only 42 studies (10%) that reflect current best practices and trends in machine learning. In the subset of 154 papers that focused on data intake and ingestion, 30 (19%) were found to incorporate the same best practices. CONCLUSION: Advances from artificial intelligence have yet to fully penetrate pharmacovigilance, although recent studies show signs that this may be changing.

Developing Crowdsourced Training Data Sets for Pharmacovigilance Intelligent Automation.

INTRODUCTION: Machine learning offers an alluring solution to developing automated approaches to the increasing individual case safety report burden being placed upon pharmacovigilance. Leveraging crowdsourcing to annotate unstructured data may provide accurate, efficient, and contemporaneous training data sets in support of machine learning. OBJECTIVE: The objective of this study was to evaluate whether crowdsourcing can be used to accurately and efficiently develop training data sets in support of pharmacovigilance automation. MATERIALS AND METHODS: Pharmacovigilance experts created a reference dataset by reviewing 15,490 de-identified social media posts of narratives pertaining to 15 drugs and 22 medically relevant topics. A random sampling of posts from the reference dataset was published on Amazon Turk and its users (Turkers) were asked a series of questions about those same medical concepts. Accuracy, price elasticity, and time efficiency were evaluated. RESULTS: Accuracy of crowdsourced curation exceeded 90% when compared to the reference dataset and was completed in about 5% of the time. There was an increase in time efficiency with higher pay, but there was no significant difference in accuracy. Additionally, having a social media post reviewed by more than one Turker (using a voting system) did not offer significant improvements in terms of accuracy. CONCLUSIONS: Crowdsourcing is an accurate and efficient method that can be used to develop training data sets in support of pharmacovigilance automation. More research is needed to better understand the breadth and depth of possible uses as well as strengths, limitations, and generalizability of results.

Using Social Listening Data to Monitor Misuse and Nonmedical Use of Bupropion: A Content Analysis.

BACKGROUND: The nonmedical use of pharmaceutical products has become a significant public health concern. Traditionally, the evaluation of nonmedical use has focused on controlled substances with addiction risk. Currently, there is no effective means of evaluating the nonmedical use of noncontrolled antidepressants. OBJECTIVE: Social listening, in the context of public health sometimes called infodemiology or infoveillance, is the process of identifying and assessing what is being said about a company, product, brand, or individual, within forms of electronic interactive media. The objectives of this study were (1) to determine whether content analysis of social listening data could be utilized to identify posts discussing potential misuse or nonmedical use of bupropion and two comparators, amitriptyline and venlafaxine, and (2) to describe and characterize these posts. METHODS: Social listening was performed on all publicly available posts cumulative through July 29, 2015, from two harm-reduction Web forums, Bluelight and Opiophile, which mentioned the study drugs. The acquired data were stripped of personally identifiable identification (PII). A set of generic, brand, and vernacular product names was used to identify product references in posts. Posts were obtained using natural language processing tools to identify vernacular references to drug misuse-related Preferred Terms from the English Medical Dictionary for Regulatory Activities (MedDRA) version 18 terminology. Posts were reviewed manually by coders, who extracted relevant details. RESULTS: A total of 7756 references to at least one of the study antidepressants were identified within posts gathered for this study. Of these posts, 668 (8.61%, 668/7756) referenced misuse or nonmedical use of the drug, with bupropion accounting for 438 (65.6%, 438/668). Of the 668 posts, nonmedical use was discouraged by 40.6% (178/438), 22% (22/100), and 18.5% (24/130) and encouraged by 12.3% (54/438), 10% (10/100), and 10.8% (14/130) for bupropion, amitriptyline, and venlafaxine, respectively. The most commonly reported desired effects were similar to stimulants with bupropion, sedatives with amitriptyline, and dissociatives with venlafaxine. The nasal route of administration was most frequently reported for bupropion, whereas the oral route was most frequently reported for amitriptyline and venlafaxine. Bupropion and venlafaxine were most commonly procured from health care providers, whereas amitriptyline was most commonly obtained or stolen from a third party. The Fleiss kappa for interrater agreement among 20 items with 7 categorical response options evaluated by all 11 raters was 0.448 (95% CI 0.421-0.457). CONCLUSIONS: Social listening, conducted in collaboration with harm-reduction Web forums, offers a valuable new data source that can be used for monitoring nonmedical use of antidepressants. Additional work on the capabilities of social listening will help further delineate the benefits and limitations of this rapidly evolving data source.

Social Media Listening for Routine Post-Marketing Safety Surveillance.

INTRODUCTION: Post-marketing safety surveillance primarily relies on data from spontaneous adverse event reports, medical literature, and observational databases. Limitations of these data sources include potential under-reporting, lack of geographic diversity, and time lag between event occurrence and discovery. There is growing interest in exploring the use of social media ('social listening') to supplement established approaches for pharmacovigilance. Although social listening is commonly used for commercial purposes, there are only anecdotal reports of its use in pharmacovigilance. Health information posted online by patients is often publicly available, representing an untapped source of post-marketing safety data that could supplement data from existing sources. OBJECTIVES: The objective of this paper is to describe one methodology that could help unlock the potential of social media for safety surveillance. METHODS: A third-party vendor acquired 24 months of publicly available Facebook and Twitter data, then processed the data by standardizing drug names and vernacular symptoms, removing duplicates and noise, masking personally identifiable information, and adding supplemental data to facilitate the review process. The resulting dataset was analyzed for safety and benefit information. RESULTS: In Twitter, a total of 6,441,679 Medical Dictionary for Regulatory Activities (MedDRA(®)) Preferred Terms (PTs) representing 702 individual PTs were discussed in the same post as a drug compared with 15,650,108 total PTs representing 946 individual PTs in Facebook. Further analysis revealed that 26 % of posts also contained benefit information. CONCLUSION: Social media listening is an important tool to augment post-marketing safety surveillance. Much work remains to determine best practices for using this rapidly evolving data source.

The support of human genetic evidence for approved drug indications.

Over a quarter of drugs that enter clinical development fail because they are ineffective. Growing insight into genes that influence human disease may affect how drug targets and indications are selected. However, there is little guidance about how much weight should be given to genetic evidence in making these key decisions. To answer this question, we investigated how well the current archive of genetic evidence predicts drug mechanisms. We found that, among well-studied indications, the proportion of drug mechanisms with direct genetic support increases significantly across the drug development pipeline, from 2.0% at the preclinical stage to 8.2% among mechanisms for approved drugs, and varies dramatically among disease areas. We estimate that selecting genetically supported targets could double the success rate in clinical development. Therefore, using the growing wealth of human genetic data to select the best targets and indications should have a measurable impact on the successful development of new drugs.

A proposed modification to Hy's law and Edish criteria in oncology clinical trials using aggregated historical data.

PURPOSE: Identifying drug-induced liver injury is a critical task in drug development and postapproval real-world care. Severe liver injury is identified by the liver chemistry threshold of alanine aminotransferase (ALT) >3× upper limit of normal (ULN) and bilirubin >2× ULN, termed Hy's law by the Food and Drug Administration. These thresholds require discontinuation of the causative drug and are seldom exceeded in most patient populations. However, because maintenance of therapy is critical in the treatment of advanced cancer, customized thresholds may be useful in oncology patient populations, particularly for those with baseline liver chemistries elevations. METHODS: Liver chemistry data from 31 aggregated oncology clinical trials were modeled through a truncated robust multivariate outlier detection (TRMOD) method to develop the decision boundary or threshold for examining liver injury in oncology clinical trials. RESULTS: The boundary of TRMOD identified outliers with an ALT limit 5.0× ULN and total bilirubin limit 2.7× ULN. In addition, TRMOD was applied to the aggregated oncology data to examine fold-baseline ALT and total bilirubin, revealing limits of ALT 6.9× baseline and bilirubin 6.5× baseline. Similar ALT and bilirubin threshold limits were observed for oncology patients both with and without liver metastases. CONCLUSIONS: These higher liver chemistry thresholds examining fold-ULN and fold-baseline data may be valuable in identifying potential severe liver injury and detecting liver safety signals of clinical concern in oncology clinical trials and postapproval settings while helping to avoid premature discontinuation of curative therapy.

Development and evaluation of a common data model enabling active drug safety surveillance using disparate healthcare databases.

OBJECTIVE: Active drug safety surveillance may be enhanced by analysis of multiple observational healthcare databases, including administrative claims and electronic health records. The objective of this study was to develop and evaluate a common data model (CDM) enabling rapid, comparable, systematic analyses across disparate observational data sources to identify and evaluate the effects of medicines. DESIGN: The CDM uses a person-centric design, with attributes for demographics, drug exposures, and condition occurrence. Drug eras, constructed to represent periods of persistent drug use, are derived from available elements from pharmacy dispensings, prescriptions written, and other medication history. Condition eras aggregate diagnoses that occur within a single episode of care. Drugs and conditions from source data are mapped to biomedical ontologies to standardize terminologies and enable analyses of higher-order effects. MEASUREMENTS: The CDM was applied to two source types: an administrative claims and an electronic medical record database. Descriptive statistics were used to evaluate transformation rules. Two case studies demonstrate the ability of the CDM to enable standard analyses across disparate sources: analyses of persons exposed to rofecoxib and persons with an acute myocardial infarction. RESULTS: Over 43 million persons, with nearly 1 billion drug exposures and 3.7 billion condition occurrences from both databases were successfully transformed into the CDM. An analysis routine applied to transformed data from each database produced consistent, comparable results. CONCLUSION: A CDM can normalize the structure and content of disparate observational data, enabling standardized analyses that are meaningfully comparable when assessing the effects of medicines.

Toward automating an inference model on unstructured terminologies: OXMIS case study.

Most modern biomedical vocabularies employ some hierarchical representation that provides a "broader/narrower" meaning relationship among the "codes" or "concepts" found within them. Often, however, we may find within the clinical setting the creation and curation of unstructured custom vocabularies used in the everyday practice of classifying and categorizing clinical data and findings.A significant and widely used example of this lies in the General Practice Research Database which makes use of the Oxford Medical Information Systems (OXMIS) coding scheme to represent drugs and medical conditions. This scheme is intrinsically unstructured, is generally regarded as disorganized, and is not amenable to comparison with other hierarchically structured medical coding schemes. To improve processes of data analysis and extraction, we define a semantically meaningful representation of the OXMIS codes by way of the Unified Medical Language System (UMLS) Metathesaurus. A structure-imposing ontology mapping is created, and this process provides a complete illustration of a general semantic mapping technique applicable to unstructured biomedical terminologies.