Towards a standard benchmark for variant and gene prioritisation algorithms: PhEval - Phenotypic inference Evaluation framework.

Background: Computational approaches to support rare disease diagnosis are challenging to build, requiring the integration of complex data types such as ontologies, gene-to-phenotype associations, and cross-species data into variant and gene prioritisation algorithms (VGPAs). However, the performance of VGPAs has been difficult to measure and is impacted by many factors, for example, ontology structure, annotation completeness or changes to the underlying algorithm. Assertions of the capabilities of VGPAs are often not reproducible, in part because there is no standardised, empirical framework and openly available patient data to assess the efficacy of VGPAs - ultimately hindering the development of effective prioritisation tools. Results: In this paper, we present our benchmarking tool, PhEval, which aims to provide a standardised and empirical framework to evaluate phenotype-driven VGPAs. The inclusion of standardised test corpora and test corpus generation tools in the PhEval suite of tools allows open benchmarking and comparison of methods on standardised data sets. Conclusions: PhEval and the standardised test corpora solve the issues of patient data availability and experimental tooling configuration when benchmarking and comparing rare disease VGPAs. By providing standardised data on patient cohorts from real-world case-reports and controlling the configuration of evaluated VGPAs, PhEval enables transparent, portable, comparable and reproducible benchmarking of VGPAs. As these tools are often a key component of many rare disease diagnostic pipelines, a thorough and standardised method of assessment is essential for improving patient diagnosis and care.

ExEmPLAR (Extracting, Exploring, and Embedding Pathways Leading to Actionable Research): a user-friendly interface for knowledge graph mining.

SUMMARY: Knowledge graphs are being increasingly used in biomedical research to link large amounts of heterogenous data and facilitate reasoning across diverse knowledge sources. Wider adoption and exploration of knowledge graphs in the biomedical research community is limited by requirements to understand the underlying graph structure in terms of entity types and relationships, represented as nodes and edges, respectively, and learn specialized query languages for graph mining and exploration. We have developed a user-friendly interface dubbed ExEmPLAR (Extracting, Exploring, and Embedding Pathways Leading to Actionable Research) to aid reasoning over biomedical knowledge graphs and assist with data-driven research and hypothesis generation. We explain the key functionalities of ExEmPLAR and demonstrate its use with a case study considering the relationship of Trypanosoma cruzi, the etiological agent of Chagas disease, to frequently associated cardiovascular conditions. AVAILABILITY AND IMPLEMENTATION: ExEmPLAR is freely accessible at https://www.exemplar.mml.unc.edu/. For code and instructions for the using the application, see: https://github.com/beasleyjonm/AOP-COP-Path-Extractor.

Integrated approach to elucidate metal-implant related adverse outcome pathways.

Exogenous metal particles and ions from implant devices are known to cause severe toxic events with symptoms ranging from adverse local tissue reactions to systemic toxicities, potentially leading to the development of cancers, heart conditions, and neurological disorders. Toxicity mechanisms, also known as Adverse Outcome Pathways (AOPs), that explain these metal-induced toxicities are severely understudied. Therefore, we deployed in silico structure- and knowledge-based approaches to identify proteome-level perturbations caused by metals and pathways that link these events to human diseases. We captured 177 structure-based, 347 knowledge-based, and 402 imputed metal-gene/protein relationships for chromium, cobalt, molybdenum, nickel, and titanium. We prioritized 72 proteins hypothesized to directly contact implant surfaces and contribute to adverse outcomes. Results of this exploratory analysis were formalized as structured AOPs. We considered three case studies reflecting the following possible situations: (i) the metal-protein-disease relationship was previously known; (ii) the metal-protein, protein-disease, and metal-disease relationships were individually known but were not linked (as a unified AOP); and (iii) one of three relationships was unknown and was imputed by our methods. These situations were illustrated by case studies on nickel-induced allergy/hypersensitivity, cobalt-induced heart failure, and titanium-induced periprosthetic osteolysis, respectively. All workflows, data, and results are freely available in https://github.com/DnlRKorn/Knowledge_Based_Metallomics/. An interactive view of select data is available at the ROBOKOP Neo4j Browser at http://robokopkg.renci.org/browser/.

PreS/MD: Predictor of Sensitization Hazard for Chemical Substances Released From Medical Devices.

In the United States, a pre-market regulatory submission for any medical device that comes into contact with either a patient or the clinical practitioner must include an adequate toxicity evaluation of chemical substances that can be released from the device during its intended use. These substances, also referred to as extractables and leachables, must be evaluated for their potential to induce sensitization/allergenicity, which traditionally has been done in animal assays such as the guinea pig maximization test (GPMT). However, advances in basic and applied science are continuously presenting opportunities to employ new approach methodologies, including computational methods which, when qualified, could replace animal testing methods to support regulatory submissions. Herein, we developed a new computational tool for rapid and accurate prediction of the GPMT outcome that we have named PreS/MD (predictor of sensitization for medical devices). To enable model development, we (1) collected, curated, and integrated the largest publicly available dataset for GPMT results; (2) succeeded in developing externally predictive (balanced accuracy of 70%-74% as evaluated by both 5-fold external cross-validation and testing of novel compounds) quantitative structure-activity relationships (QSAR) models for GPMT using machine learning algorithms, including deep learning; and (3) developed a publicly accessible web portal integrating PreS/MD models that can predict GPMT outcomes for any molecule of interest. We expect that PreS/MD will be used by both industry and regulatory scientists in medical device safety assessments and help replace, reduce, or refine the use of animals in toxicity testing. PreS/MD is freely available at https://presmd.mml.unc.edu/.

STopTox: An in Silico Alternative to Animal Testing for Acute Systemic and Topical Toxicity.

BACKGROUND: Modern chemical toxicology is facing a growing need to Reduce, Refine, and Replace animal tests (Russell 1959) for hazard identification. The most common type of animal assays for acute toxicity assessment of chemicals used as pesticides, pharmaceuticals, or in cosmetic products is known as a "6-pack" battery of tests, including three topical (skin sensitization, skin irritation and corrosion, and eye irritation and corrosion) and three systemic (acute oral toxicity, acute inhalation toxicity, and acute dermal toxicity) end points. METHODS: We compiled, curated, and integrated, to the best of our knowledge, the largest publicly available data sets and developed an ensemble of quantitative structure-activity relationship (QSAR) models for all six end points. All models were validated according to the Organisation for Economic Co-operation and Development (OECD) QSAR principles, using data on compounds not included in the training sets. RESULTS: In addition to high internal accuracy assessed by cross-validation, all models demonstrated an external correct classification rate ranging from 70% to 77%. We established a publicly accessible Systemic and Topical chemical Toxicity (STopTox) web portal (https://stoptox.mml.unc.edu/) integrating all developed models for 6-pack assays. CONCLUSIONS: We developed STopTox, a comprehensive collection of computational models that can be used as an alternative to in vivo 6-pack tests for predicting the toxicity hazard of small organic molecules. Models were established following the best practices for the development and validation of QSAR models. Scientists and regulators can use the STopTox portal to identify putative toxicants or nontoxicants in chemical libraries of interest. https://doi.org/10.1289/EHP9341.

ZINC Express: A Virtual Assistant for Purchasing Compounds Annotated in the ZINC Database.

Many laboratories working in the field of drug discovery use the ZINC database to identify and then acquire commercially available chemicals. However, finding the best deal for a given compound is often time-intensive and laborious, as the process involves searching for all vendors selling the desired compound, comparing prices, and interacting with the preferred vendor. To streamline this process, we have developed ZINC Express, a web application that simplifies the online purchase of chemicals annotated in the ZINC database. For any compound with a known ZINC ID, ZINC Express finds a list of vendors offering that compound and for each such vendor returns the available package quantities, the price of each package, and the price per milligram along with a link to that vendor. We expect that ZINC Express will be of use to both computational and experimental researchers. ZINC Express is freely accessible online at https://zincexpress.mml.unc.edu/.