Best practice and reproducible science are required to advance artificial intelligence in real-world applications.

Drug-induced liver injury (DILI) is one of the most significant concerns in medical practice but yet it still cannot be fully recapitulated with existing in vivo, in vitro and in silico approaches. To address this challenge, Chen et al. [ 1] developed a deep learning-based DILI prediction model based on chemical structure information alone. The reported model yielded an outstanding prediction performance (i.e. 0.958, 0.976, 0.935, 0.947, 0.926 and 0.913 for AUC, accuracy, recall, precision, F1-score and specificity, respectively, on a test set), far outperforming all publicly available and similar in silico DILI models. This extraordinary model performance is counter-intuitive to what we know about the underlying biology of DILI and the principles and hypothesis behind this type of in silico approach. In this Letter to the Editor, we raise awareness of several issues concerning data curation, model validation and comparison practices, and data and model reproducibility.

Generation of a drug-induced renal injury list to facilitate the development of new approach methodologies for nephrotoxicity.

Drug-induced renal injury (DIRI) causes >1.5 million adverse events annually in the USA alone. Although standard biomarkers exist for DIRI, they lack the sensitivity or specificity to detect nephrotoxicity before the significant loss of renal function. In this study, we describe the creation of DIRIL - a list of drugs associated with DIRI and nephrotoxicity - from two literature datasets with DIRI annotation, confirmed using FDA drug labeling. DIRIL comprises 317 orally administered drugs covering all 14 anatomical, therapeutic and chemical (ATC) classification categories. Of the 317 drugs, 171 were DIRI-positive and 146 were DIRI-negative. DIRIL will be a relevant and invaluable resource for discovery of new approach methods (NAMs) to predict the occurrence and possible severity of DIRI earlier in drug development.

Adaptability of AI for safety evaluation in regulatory science: A case study of drug-induced liver injury.

Artificial intelligence (AI) has played a crucial role in advancing biomedical sciences but has yet to have the impact it merits in regulatory science. As the field advances, in silico and in vitro approaches have been evaluated as alternatives to animal studies, in a drive to identify and mitigate safety concerns earlier in the drug development process. Although many AI tools are available, their acceptance in regulatory decision-making for drug efficacy and safety evaluation is still a challenge. It is a common perception that an AI model improves with more data, but does reality reflect this perception in drug safety assessments? Importantly, a model aiming at regulatory application needs to take a broad range of model characteristics into consideration. Among them is adaptability, defined as the adaptive behavior of a model as it is retrained on unseen data. This is an important model characteristic which should be considered in regulatory applications. In this study, we set up a comprehensive study to assess adaptability in AI by mimicking the real-world scenario of the annual addition of new drugs to the market, using a model we previously developed known as DeepDILI for predicting drug-induced liver injury (DILI) with a novel Deep Learning method. We found that the target test set plays a major role in assessing the adaptive behavior of our model. Our findings also indicated that adding more drugs to the training set does not significantly affect the predictive performance of our adaptive model. We concluded that the proposed adaptability assessment framework has utility in the evaluation of the performance of a model over time.

DICE: A Drug Indication Classification and Encyclopedia for AI-Based Indication Extraction.

Drug labeling contains an 'INDICATIONS AND USAGE' that provides vital information to support clinical decision making and regulatory management. Effective extraction of drug indication information from free-text based resources could facilitate drug repositioning projects and help collect real-world evidence in support of secondary use of approved medicines. To enable AI-powered language models for the extraction of drug indication information, we used manual reading and curation to develop a Drug Indication Classification and Encyclopedia (DICE) based on FDA approved human prescription drug labeling. A DICE scheme with 7,231 sentences categorized into five classes (indications, contradictions, side effects, usage instructions, and clinical observations) was developed. To further elucidate the utility of the DICE, we developed nine different AI-based classifiers for the prediction of indications based on the developed DICE to comprehensively assess their performance. We found that the transformer-based language models yielded an average MCC of 0.887, outperforming the word embedding-based Bidirectional long short-term memory (BiLSTM) models (0.862) with a 2.82% improvement on the test set. The best classifiers were also used to extract drug indication information in DrugBank and achieved a high enrichment rate (>0.930) for this task. We found that domain-specific training could provide more explainable models without performance sacrifices and better generalization for external validation datasets. Altogether, the proposed DICE could be a standard resource for the development and evaluation of task-specific AI-powered, natural language processing (NLP) models.

Three Complete Genome Sequences of Genotype G Mumps Virus from the 2016 Outbreak in Arkansas, USA.

We present here the complete genome sequences of three mumps virus (MuV) strains isolated from patients who tested positive for the mumps virus during a mumps outbreak in Springdale, AR (USA), in 2016. The virus genomes, sequenced with Oxford Nanopore technology, belong to genotype G and have an average length of 15,342 nucleotides (nt).