Comparison of the MOdified NARanjo Causality Scale (MONARCSi) for Individual Case Safety Reports vs. a Reference Standard.

INTRODUCTION: In 2018, we published the MONARCSi algorithmic decision support tool showing high inter-rater agreement, moderate sensitivity, and high specificity compared with drug-event pairs (DEPs) previously reviewed using current, industry-established approaches. Following publication, MONARCSi was implemented as a prototype system to facilitate medical review of individual case safety reports (ICSRs). This paper presents subsequent evaluation of MONARCSi-supported causality assessments against an independent, best achievable reference standard. OBJECTIVE: This paper describes the development of an independent reference standard (i.e., reference comparator) using a sample of DEPs evaluated by Roche subject matter experts (SMEs) and subsequent performance analysis for both the reference standard and MONARCSi. METHODS: Roche collected a random sample of 131 DEPs evaluated by an external vendor using the MONARCSi prototype during 2020, and collectively referred to as the VMON (Vendor using the MONARCSi system for medical review) dataset. An internal group of causality SMEs (aka CAUSMET) were recruited and trained to assess the same DEPs independently using the MONARCSi structure with Global Introspection to determine their individual assessments of causality. The CAUSMET final causality was determined using a majority voting rule. RESULTS: Binary comparison of the aggregate results showed substantial agreement (Gwet kappa = 0.81) between the VMON and reference standard CAUSMET assessments. Bayesian latent class modeling showed that both the reference standard and VMON assessments exhibited similar high posterior mean sensitivity and specificity (CAUSMET: 89 and 93%, respectively; VMON: 87 and 94%, respectively). Finally, comparison of the sensitivity and specificity suggested no obvious difference across groups. CONCLUSION: Analysis of causality results from the assessments by independent internal SMEs using MONARCSi shows there is no obvious difference in performance between the aggregate CAUSMET and VMON assessments based on the comparison of specificity and sensitivity. These results further support use of MONARCSi as a decision support tool for evaluating drug-event causality in a consistent and documentable manner.

Industry Assessment of the Contribution of Patient Support Programs, Market Research Programs, and Social Media to Patient Safety.

BACKGROUND: Over the past decade, the volume of adverse events (AEs) reported to marketing authorization holders and regulators has been rapidly increasing each year, which has led to significant challenges in patient safety assessment. Three data sources that have largely contributed to the expansion in adverse event reports are patient support programs (PSPs), market research programs (MRPs), and social media. In this study, we sought to further understand the contribution of these safety data sources to the characterization of a product's safety profile. METHODS: Three separate approaches were taken that, when combined, can be used to evaluate each data source. The first identified any core company data sheet changes or drug safety label changes. The second evaluated the similarity of information through proportions of AEs between each solicited data source and spontaneous sources. Lastly, the completeness of information reported was evaluated through vigiGrade and compared across each data source. RESULTS: One drug safety label change was identified from a patient support program, which involved regular contact with health care providers. No label changes were identified from market research programs or social media. Patient support programs, market research programs, and social media report similar proportions for HLGT as spontaneous sources. Market research programs and social media display very low vigiGrade scores. When broken down by subtype, traditional PSPs display high vigiGrade scores, while patient assistance programs display lower vigiGrade scores that were program dependent. CONCLUSIONS: This study did not demonstrate that certain data sources such as market research programs, social media, and patient assistance programs meaningfully contributed to the further understanding of the characterization of a product's safety profile.

A Study on the Application and Use of Artificial Intelligence to Support Drug Development.

PURPOSE: The Tufts Center for the Study of Drug Development (CSDD) and the Drug Information Association (DIA) in collaboration with 8 pharmaceutical and biotechnology companies conducted a study examining the adoption and effect of artificial intelligence (AI), such as machine learning, on drug development. The study was conducted to clarify and understand AI adoption across the industry and to gather detailed insights into the spectrum of activities included in the definition of AI. The study investigated and identified analytical platforms and innovations across pharmaceutical and biotechnology companies currently being used or planned for in the future. METHODS: A 2-part method was used that comprised in-depth interviews with AI industry experts and a global survey conducted across pharmaceutical and biotechnology organizations. Eleven in-depth interviews focused on use and implementation of AI across drug development. The survey assessed use of AI and included perceptions about current and future use. The survey also examined technology definitions, assessment of organizational and personal AI expertise, and use of partnerships. A total of 402 responses, including data from 217 unique organizations, were analyzed. FINDINGS: Although 7 in 10 respondents reported using AI in some capacity, a wide range of use was reported by AI type. Patient selection and recruitment for clinical studies was the most commonly reported AI activity, with 34 respondents currently using AI for this activity. In addition, identification of medicinal products data gathering was the top activity being piloted or in the planning stages, reported by 49 respondents. The study also revealed that the most significant challenges to AI implementation included staff skills (55%), data structure (52%), and budgets (49%). Nearly 60% of respondents noted planned increases in staff within 1-2 years to support AI use or implementation. IMPLICATIONS: Despite the challenges to AI implementation, the survey revealed that most organizations use AI in some capacity and that it is important to the success of an organization's workforce. Many organizations reported expectations for increasing staff as implementation of AI expands. Further research should examine the changing development landscape as the role of AI evolves.

Advancing nonclinical innovation and safety in pharmaceutical testing.

Nonclinical tests are considered crucial for understanding the safety of investigational medicines. However, the effective translation from nonclinical to human application is limited and must be improved. Drug development stakeholders are working to advance human-based in vitro and in silico methods that may be more predictive of human efficacy and safety in vivo because they enable scientists to model the direct interaction of drugs with human cells, tissues, and biological processes. Here, we recommend test-neutral regulations; increased funding for development and integration of human-based approaches; support for existing initiatives that advance human-based approaches; evaluation of new approaches using human data; establishment of guidelines for procuring human cells and tissues for research; and additional training and educational opportunities in human-based approaches.

Pharmacovigilance is Everyone's Concern: Let's Work It Out Together.

In recent decades, the field of drug safety/pharmacovigilance (PV) has advanced dramatically in some ways and yet has remained stagnant or progressed slowly in others. One way to assess the PV landscape is to view it through both a regulatory lens and a science and technology lens. This commentary highlights some of the current PV issues that can be resolved by sustained collaboration among all relevant stakeholders.