A framework enabling LLMs into regulatory environment for transparency and trustworthiness and its application to drug labeling document.

Regulatory agencies consistently deal with extensive document reviews, ranging from product submissions to both internal and external communications. Large Language Models (LLMs) like ChatGPT can be invaluable tools for these tasks, however present several challenges, particularly the proprietary information, combining customized function with specific review needs, and transparency and explainability of the model's output. Hence, a localized and customized solution is imperative. To tackle these challenges, we formulated a framework named askFDALabel on FDA drug labeling documents that is a crucial resource in the FDA drug review process. AskFDALabel operates within a secure IT environment and comprises two key modules: a semantic search and a Q&A/text-generation module. The Module S built on word embeddings to enable comprehensive semantic queries within labeling documents. The Module T utilizes a tuned LLM to generate responses based on references from Module S. As the result, our framework enabled small LLMs to perform comparably to ChatGPT with as a computationally inexpensive solution for regulatory application. To conclude, through AskFDALabel, we have showcased a pathway that harnesses LLMs to support agency operations within a secure environment, offering tailored functions for the needs of regulatory research.

Tocotrienols Provide Radioprotection to Multiple Organ Systems through Complementary Mechanisms of Antioxidant and Signaling Effects.

Tocotrienols have powerful radioprotective properties in multiple organ systems and are promising candidates for development as clinically effective radiation countermeasures. To facilitate their development as clinical radiation countermeasures, it is crucial to understand the mechanisms behind their powerful multi-organ radioprotective properties. In this context, their antioxidant effects are recognized for directly preventing oxidative damage to cellular biomolecules from ionizing radiation. However, there is a growing body of evidence indicating that the radioprotective mechanism of action for tocotrienols extends beyond their antioxidant properties. This raises a new pharmacological paradigm that tocotrienols are uniquely efficacious radioprotectors due to a synergistic combination of antioxidant and other signaling effects. In this review, we have covered the wide range of multi-organ radioprotective effects observed for tocotrienols and the mechanisms underlying it. These radioprotective effects for tocotrienols can be characterized as (1) direct cytoprotective effects, characteristic of the classic antioxidant properties, and (2) other effects that modulate a wide array of critical signaling factors involved in radiation injury.

DeepAmes: A deep learning-powered Ames test predictive model with potential for regulatory application.

The Ames assay is required by the regulatory agencies worldwide to assess the mutagenic potential risk of consumer products. As well as this in vitro assay, in silico approaches have been widely used to predict Ames test results as outlined in the International Council for Harmonization (ICH) guidelines. Building on this in silico approach, here we describe DeepAmes, a high performance and robust model developed with a novel deep learning (DL) approach for potential utility in regulatory science. DeepAmes was developed with a large and consistent Ames dataset (>10,000 compounds) and was compared with other five standard Machine Learning (ML) methods. Using a test set of 1,543 compounds, DeepAmes was the best performer in predicting the outcome of Ames assay. In addition, DeepAmes yielded the best and most stable performance up to when compounds were >30% outside of the applicability domain (AD). Regarding the potential for regulatory application, a revised version of DeepAmes with a much-improved sensitivity of 0.87 from 0.47. In conclusion, DeepAmes provides a DL-powered Ames test predictive model for predicting the results of Ames tests; with its defined AD and clear context of use, DeepAmes has potential for utility in regulatory application.

Artificial intelligence and real-world data for drug and food safety - A regulatory science perspective.

In 2013, the Global Coalition for Regulatory Science Research (GCRSR) was established with members from over ten countries (www.gcrsr.net). One of the main objectives of GCRSR is to facilitate communication among global regulators on the rise of new technologies with regulatory applications through the annual conference Global Summit on Regulatory Science (GSRS). The 11th annual GSRS conference (GSRS21) focused on "Regulatory Sciences for Food/Drug Safety with Real-World Data (RWD) and Artificial Intelligence (AI)." The conference discussed current advancements in both AI and RWD approaches with a specific emphasis on how they impact regulatory sciences and how regulatory agencies across the globe are pursuing the adaptation and oversight of these technologies. There were presentations from Brazil, Canada, India, Italy, Japan, Germany, Switzerland, Singapore, the United Kingdom, and the United States. These presentations highlighted how various agencies are moving forward with these technologies by either improving the agencies' operation and/or preparing regulatory mechanisms to approve the products containing these innovations. To increase the content and discussion, the GSRS21 hosted two debate sessions on the question of "Is Regulatory Science Ready for AI?" and a workshop to showcase the analytical data tools that global regulatory agencies have been using and/or plan to apply to regulatory science. Several key topics were highlighted and discussed during the conference, such as the capabilities of AI and RWD to assist regulatory science policies for drug and food safety, the readiness of AI and data science to provide solutions for regulatory science. Discussions highlighted the need for a constant effort to evaluate emerging technologies for fit-for-purpose regulatory applications. The annual GSRS conferences offer a unique platform to facilitate discussion and collaboration across regulatory agencies, modernizing regulatory approaches, and harmonizing efforts.

Corrigendum: DeepCarc: Deep learning-powered carcinogenicity prediction using model-level representation.

[This corrects the article DOI: 10.3389/frai.2021.757780.].

Adaptability of AI for safety evaluation in regulatory science: A case study of drug-induced liver injury.

Artificial intelligence (AI) has played a crucial role in advancing biomedical sciences but has yet to have the impact it merits in regulatory science. As the field advances, in silico and in vitro approaches have been evaluated as alternatives to animal studies, in a drive to identify and mitigate safety concerns earlier in the drug development process. Although many AI tools are available, their acceptance in regulatory decision-making for drug efficacy and safety evaluation is still a challenge. It is a common perception that an AI model improves with more data, but does reality reflect this perception in drug safety assessments? Importantly, a model aiming at regulatory application needs to take a broad range of model characteristics into consideration. Among them is adaptability, defined as the adaptive behavior of a model as it is retrained on unseen data. This is an important model characteristic which should be considered in regulatory applications. In this study, we set up a comprehensive study to assess adaptability in AI by mimicking the real-world scenario of the annual addition of new drugs to the market, using a model we previously developed known as DeepDILI for predicting drug-induced liver injury (DILI) with a novel Deep Learning method. We found that the target test set plays a major role in assessing the adaptive behavior of our model. Our findings also indicated that adding more drugs to the training set does not significantly affect the predictive performance of our adaptive model. We concluded that the proposed adaptability assessment framework has utility in the evaluation of the performance of a model over time.

Best practice and reproducible science are required to advance artificial intelligence in real-world applications.

Drug-induced liver injury (DILI) is one of the most significant concerns in medical practice but yet it still cannot be fully recapitulated with existing in vivo, in vitro and in silico approaches. To address this challenge, Chen et al. [ 1] developed a deep learning-based DILI prediction model based on chemical structure information alone. The reported model yielded an outstanding prediction performance (i.e. 0.958, 0.976, 0.935, 0.947, 0.926 and 0.913 for AUC, accuracy, recall, precision, F1-score and specificity, respectively, on a test set), far outperforming all publicly available and similar in silico DILI models. This extraordinary model performance is counter-intuitive to what we know about the underlying biology of DILI and the principles and hypothesis behind this type of in silico approach. In this Letter to the Editor, we raise awareness of several issues concerning data curation, model validation and comparison practices, and data and model reproducibility.

DeepCarc: Deep Learning-Powered Carcinogenicity Prediction Using Model-Level Representation.

Carcinogenicity testing plays an essential role in identifying carcinogens in environmental chemistry and drug development. However, it is a time-consuming and label-intensive process to evaluate the carcinogenic potency with conventional 2-years rodent animal studies. Thus, there is an urgent need for alternative approaches to providing reliable and robust assessments on carcinogenicity. In this study, we proposed a DeepCarc model to predict carcinogenicity for small molecules using deep learning-based model-level representations. The DeepCarc Model was developed using a data set of 692 compounds and evaluated on a test set containing 171 compounds in the National Center for Toxicological Research liver cancer database (NCTRlcdb). As a result, the proposed DeepCarc model yielded a Matthews correlation coefficient (MCC) of 0.432 for the test set, outperforming four advanced deep learning (DL) powered quantitative structure-activity relationship (QSAR) models with an average improvement rate of 37%. Furthermore, the DeepCarc model was also employed to screen the carcinogenicity potential of the compounds from both DrugBank and Tox21. Altogether, the proposed DeepCarc model could serve as an early detection tool (https://github.com/TingLi2016/DeepCarc) for carcinogenicity assessment.

Development of a Battery of In Silico Prediction Tools for Drug-Induced Liver Injury from the Vantage Point of Translational Safety Assessment.

Drug-induced liver injury (DILI) remains a challenge when translating knowledge from the preclinical stage to human use cases. Attempts to model human DILI directly based on the information from drug labels have had some success; however, the approach falls short of providing insights or addressing uncertainty due to the difficulty of decoupling the idiosyncratic nature of human DILI outcomes. Our approach in this comparative analysis is to leverage existing preclinical and clinical data as well as information on metabolism to better translate mammalian to human DILI. The human DILI knowledge base from the United States Food and Drug Administration (U.S. FDA) National Center for Toxicology Research contains 1036 pharmaceuticals from diverse therapeutic categories. A human DILI training set of 305 oral marketed drugs was prepared and a binary classification scheme applied. The second knowledge base consists of mammalian repeated dose toxicity with liver toxicity data from various regulatory sources. Within this knowledge base, we identified 278 pharmaceuticals containing 198 marketed or withdrawn oral drugs with data from the U.S. FDA new drug application and 98 active pharmaceutical ingredients from ToxCast. From this collection, a set of 225 oral drugs was prepared as the mammalian hepatotoxicity training set with particular end points of pathology findings in the liver and bile duct. Both human and mammalian data sets were processed using various learning algorithms, including artificial intelligence approaches. The external validations for both models were comparable to the training statistics. These data sets were also used to extract species-differentiating chemotypes that differentiate DILI effects on humans from mammals. A systematic workflow was devised to predict human DILI and provide mechanistic insights. For a given query molecule, both human and mammalian models are run. If the predictions are discordant, both metabolites and parents are investigated for quantitative structure-activity relationship and species-differentiating chemotypes. Their results are combined using the Dempster-Shafer decision theory to yield a final outcome prediction for human DILI with estimated uncertainty. Finally, these tools are implementable within an in silico platform for systematic evaluation.

DeepDILI: Deep Learning-Powered Drug-Induced Liver Injury Prediction Using Model-Level Representation.

Drug-induced liver injury (DILI) is the most frequently reported single cause of safety-related withdrawal of marketed drugs. It is essential to identify drugs with DILI potential at the early stages of drug development. In this study, we describe a deep learning-powered DILI (DeepDILI) prediction model created by combining model-level representation generated by conventional machine learning (ML) algorithms with a deep learning framework based on Mold2 descriptors. We conducted a comprehensive evaluation of the proposed DeepDILI model performance by posing several critical questions: (1) Could the DILI potential of newly approved drugs be predicted by accumulated knowledge of early approved ones? (2) is model-level representation more informative than molecule-based representation for DILI prediction? and (3) could improved model explainability be established? For question 1, we developed the DeepDILI model using drugs approved before 1997 to predict the DILI potential of those approved thereafter. As a result, the DeepDILI model outperformed the five conventional ML algorithms and two state-of-the-art ensemble methods with a Matthews correlation coefficient (MCC) value of 0.331. For question 2, we demonstrated that the DeepDILI model's performance was significantly improved (i.e., a MCC improvement of 25.86% in test set) compared with deep neural networks based on molecule-based representation. For question 3, we found 21 chemical descriptors that were enriched, suggesting a strong association with DILI outcome. Furthermore, we found that the DeepDILI model has more discrimination power to identify the DILI potential of drugs belonging to the World Health Organization therapeutic category of 'alimentary tract and metabolism'. Moreover, the DeepDILI model based on Mold2 descriptors outperformed the ones with Mol2vec and MACCS descriptors. Finally, the DeepDILI model was applied to the recent real-world problem of predicting any DILI concern for potential COVID-19 treatments from repositioning drug candidates. Altogether, this developed DeepDILI model could serve as a promising tool for screening for DILI risk of compounds in the preclinical setting, and the DeepDILI model is publicly available through https://github.com/TingLi2016/DeepDILI.

Deep Learning on High-Throughput Transcriptomics to Predict Drug-Induced Liver Injury.

Drug-induced liver injury (DILI) is one of the most cited reasons for the high drug attrition rate and drug withdrawal from the market. The accumulated large amount of high throughput transcriptomic profiles and advances in deep learning provide an unprecedented opportunity to improve the suboptimal performance of DILI prediction. In this study, we developed an eight-layer Deep Neural Network (DNN) model for DILI prediction using transcriptomic profiles of human cell lines (LINCS L1000 dataset) with the current largest binary DILI annotation data [i.e., DILI severity and toxicity (DILIst)]. The developed models were evaluated by Monte Carlo cross-validation (MCCV), permutation test, and an independent validation (IV) set. The developed DNN model achieved the area under the receiver operating characteristic curve (AUC) of 0.802 and 0.798, and balanced accuracy of 0.741 and 0.721 for training and an IV set, respectively, outperforming the conventional machine learning algorithms, including K-nearest neighbors (KNN), Support Vector Machine (SVM), and Random Forest (RF). Moreover, the developed DNN model provided a more balanced sensitivity of 0.839 and specificity of 0.603. Besides, we found the developed DNN model had a superior predictive performance for oncology drugs. Also, the functional and network analysis of genes driving the predictions revealed their relevance to the underlying mechanisms of DILI. The proposed DNN model could be a promising tool for early detection of DILI potential in the pre-clinical setting.

Linking Pharmacogenomic Information on Drug Safety and Efficacy with Ethnic Minority Populations.

Numerous prescription drugs' labeling contains pharmacogenomic (PGx) information to aid health providers and patients in the safe and effective use of drugs. However, clinical studies for such PGx biomarkers and related drug doses are generally not conducted in diverse ethnic populations. Thus, it is urgently important to incorporate PGx information with genetic characteristics of racial and ethnic minority populations and utilize it to promote minority health. In this project a bioinformatics approach was developed to enhance the collection of PGx information related to ethnic minorities to pave the way toward understanding the population-wide utility of PGx information. To address this challenge, we first gathered PGx information from drug labels. Second, we extracted data on the allele frequency information of genetic variants in ethnic minority groups from public resources. Then, we collected published research articles on PGx biomarkers and related drugs for reference. Finally, the data were integrated and formatted to build a new PGx database containing information on known drugs and biomarkers for ethnic minority groups. This database provides scientific information needed to evaluate available PGx information to enhance drug dose selection and drug safety for ethnic minority populations.

Imaging the In Vivo Degradation of Tissue Engineering Implants by Use of Supramolecular Radiopaque Biomaterials.

For in situ tissue engineering (TE) applications it is important that implant degradation proceeds in concord with neo-tissue formation to avoid graft failure. It will therefore be valuable to have an imaging contrast agent (CA) available that can report on the degrading implant. For this purpose, a biodegradable radiopaque biomaterial is presented, modularly composed of a bisurea chain-extended polycaprolactone (PCL2000-U4U) elastomer and a novel iodinated bisurea-modified CA additive (I-U4U). Supramolecular hydrogen bonding interactions between the components ensure their intimate mixing. Porous implant TE-grafts are prepared by simply electrospinning a solution containing PCL2000-U4U and I-U4U. Rats receive an aortic interposition graft, either composed of only PCL2000-U4U (control) or of PCL2000-U4U and I-U4U (test). The grafts are explanted for analysis at three time points over a 1-month period. Computed tomography imaging of the test group implants prior to explantation shows a decrease in iodide volume and density over time. Explant analysis also indicates scaffold degradation. (Immuno)histochemistry shows comparable cellular contents and a similar neo-tissue formation process for test and control group, demonstrating that the CA does not have apparent adverse effects. A supramolecular approach to create solid radiopaque biomaterials can therefore be used to noninvasively monitor the biodegradation of synthetic implants.

Regulatory landscape of dietary supplements and herbal medicines from a global perspective.

The number of Individuals that use dietary supplements and herbal medicine products are continuous to increase in many countries. The context of usage of a dietary supplement varies widely from country-to-country; in some countries supplement use is just limited to general health and well-being while others permit use for medicinal purposes. To date, there is little consensus from country to country on the scope, requirements, definition, or even the terminology in which dietary supplement and herbal medicines categories could be classified. Transparent science-based quality standards for the ingredients across these regulatory frameworks/definitions becomes even more important given the international supply chain. Meanwhile, there has been a rapid advancement in emerging technologies and data science applied to the field. This review was conceived at the Global Summit on Regulatory Sciences that took place in Beijing on September 2018 (GSRS2018) which is organized by Global Coalition for Regulatory Science Research (GCRSR) that consists of the global regulatory agencies from over ten countries including the European Union. This review summarizes a significant portion of discussions relating to a longitudinal comparison of the status for dietary supplements and herbal medicines among the different national jurisdictions and to the extent of how new tools and methodologies can improve the regulatory application.

Study of pharmacogenomic information in FDA-approved drug labeling to facilitate application of precision medicine.

Pharmacogenomics (PGx), studying the relationship between drug response and genetic makeup of an individual, is accelerating advances in precision medicine. The FDA includes PGx information in the labeling of approved drugs to better inform on their safety and effectiveness. We herein present a summary of PGx information found in 261 prescription drug labeling documents by querying the publicly available FDALabel database. A total of 362 drug-biomarker pairs (DBPs) were identified. We profiled DBPs using frequency of the biomarkers and their therapeutic classes. Four categories of applications (indication, safety, dosing and information) were discussed according to information in labeling. This analysis facilitates better understanding, utilization and translation of PGx information in drug labeling among researchers, healthcare professionals and the public.

Can Transcriptomic Profiles from Cancer Cell Lines Be Used for Toxicity Assessment?

In vitro toxicogenomics (TGx) has the potential to replace or supplement animal studies. However, TGx studies often suffer from a limited sample size and cell types. Meanwhile, transcriptomic data have been generated for tens of thousands of compounds using cancer cell lines mainly for drug efficacy screening. Here, we asked the question of whether these types of transcriptomic data can be used to support toxicity assessment. We compared transcriptomic profiles from three cancer lines (HL60, MCF7, and PC3) from the CMap data set with those using primary hepatocytes or in vivo repeated dose studies from the Open TG-GATEs database by using our previously reported pair ranking (PRank) method. We observed an encouraging similarity between HL60 and human primary hepatocytes (PRank score = 0.70), suggesting the two cellular assays could be potentially interchangeable. When the analysis was limited to drug-induced liver injury (DILI)-related compounds or genes, the cancer cell lines exhibited promise in DILI assessment in comparison with conventional TGx systems (i.e., human primary hepatocytes or rat in vivo repeated dose). Also, some toxicity-related pathways, such as PPAR signaling pathways and fatty acid-related pathways, were preserved across various assay systems, indicating the assay transferability is biological process-specific. Furthermore, we established a potential application of transcriptomic profiles of cancer cell lines for studying immune-related biological processes involving some specific cell types. Moreover, if PRank analysis was focused on only landmark genes from L1000 or S1500+, the advantage of cancer cell lines over the TGx studies was limited. In conclusion, repurposing of existing cancer-related transcript profiling data has great potential for toxicity assessment, particularly in predicting DILI.

Drug-induced liver injury severity and toxicity (DILIst): binary classification of 1279 drugs by human hepatotoxicity.

Drug-induced liver injury (DILI) is of significant concern to drug development and regulatory review because of the limited success with existing preclinical models. For developing alternative methods, a large drug list is needed with known DILI severity and toxicity. We augmented the DILIrank data set [annotated using US Food and Drug Administration (FDA) drug labeling)] with four literature datasets (N >350 drugs) to generate the largest drug list with DILI classification, called DILIst (DILI severity and toxicity). DILIst comprises 1279 drugs, of which 768 were DILI positives (increase of 65% from DILIrank), whereas 511 were DILI negatives (increase of 65%). The investigation of DILI positive-negative distribution across various therapeutic categories revealed the most and least frequent DILI categories. Thus, we consider DILIst to be an invaluable resource for the community to improve DILI research.

Study of serious adverse drug reactions using FDA-approved drug labeling and MedDRA.

BACKGROUND: Adverse Drug Reactions (ADRs) are of great public health concern. FDA-approved drug labeling summarizes ADRs of a drug product mainly in three sections, i.e., Boxed Warning (BW), Warnings and Precautions (WP), and Adverse Reactions (AR), where the severity of ADRs are intended to decrease in the order of BW > WP > AR. Several reported studies have extracted ADRs from labeling documents, but most, if not all, did not discriminate the severity of the ADRs by the different labeling sections. Such a practice could overstate or underestimate the impact of certain ADRs to the public health. In this study, we applied the Medical Dictionary for Regulatory Activities (MedDRA) to drug labeling and systematically analyzed and compared the ADRs from the three labeling sections with a specific emphasis on analyzing serious ADRs presented in BW, which is of most drug safety concern. RESULTS: This study investigated New Drug Application (NDA) labeling documents for 1164 single-ingredient drugs using Oracle Text search to extract MedDRA terms. We found that only a small portion of MedDRA Preferred Terms (PTs), 3819 out of 21,920 or 17.42%, were observed in a whole set of documents. In detail, 466/3819 (12.0%) PTs were in BW, 2023/3819 (53.0%) were in WP, and 2961/3819 (77.5%) were in AR sections. We also found a higher overlap of top 20 occurring BW PTs with WP sections compared to AR sections. Within the MedDRA System Organ Class levels, serious ADRs (sADRs) from BW were prevalent in Nervous System disorders and Vascular disorders. A Hierarchical Cluster Analysis (HCA) revealed that drugs within the same therapeutic category shared the same ADR patterns in BW (e.g., nervous system drug class is highly associated with drug abuse terms such as dependence, substance abuse, and respiratory depression). CONCLUSIONS: This study demonstrated that combining MedDRA standard terminologies with data mining techniques facilitated computer-aided ADR analysis of drug labeling. We also highlighted the importance of labeling sections that differ in seriousness and application in drug safety. Using sADRs primarily related to BW sections, we illustrated a prototype approach for computer-aided ADR monitoring and studies which can be applied to other public health documents.