Results and lessons learned from the sbv IMPROVER metagenomics diagnostics for inflammatory bowel disease challenge.

A growing body of evidence links gut microbiota changes with inflammatory bowel disease (IBD), raising the potential benefit of exploiting metagenomics data for non-invasive IBD diagnostics. The sbv IMPROVER metagenomics diagnosis for inflammatory bowel disease challenge investigated computational metagenomics methods for discriminating IBD and nonIBD subjects. Participants in this challenge were given independent training and test metagenomics data from IBD and nonIBD subjects, which could be wither either raw read data (sub-challenge 1, SC1) or processed Taxonomy- and Function-based profiles (sub-challenge 2, SC2). A total of 81 anonymized submissions were received between September 2019 and March 2020. Most participants' predictions performed better than random predictions in classifying IBD versus nonIBD, Ulcerative Colitis (UC) versus nonIBD, and Crohn's Disease (CD) versus nonIBD. However, discrimination between UC and CD remains challenging, with the classification quality similar to the set of random predictions. We analyzed the class prediction accuracy, the metagenomics features by the teams, and computational methods used. These results will be openly shared with the scientific community to help advance IBD research and illustrate the application of a range of computational methodologies for effective metagenomic classification.

Crowdsourced benchmarking of taxonomic metagenome profilers: lessons learned from the sbv IMPROVER Microbiomics challenge.

BACKGROUND: Selection of optimal computational strategies for analyzing metagenomics data is a decisive step in determining the microbial composition of a sample, and this procedure is complex because of the numerous tools currently available. The aim of this research was to summarize the results of crowdsourced sbv IMPROVER Microbiomics Challenge designed to evaluate the performance of off-the-shelf metagenomics software as well as to investigate the robustness of these results by the extended post-challenge analysis. In total 21 off-the-shelf taxonomic metagenome profiling pipelines were benchmarked for their capacity to identify the microbiome composition at various taxon levels across 104 shotgun metagenomics datasets of bacterial genomes (representative of various microbiome samples) from public databases. Performance was determined by comparing predicted taxonomy profiles with the gold standard. RESULTS: Most taxonomic profilers performed homogeneously well at the phylum level but generated intermediate and heterogeneous scores at the genus and species levels, respectively. kmer-based pipelines using Kraken with and without Bracken or using CLARK-S performed best overall, but they exhibited lower precision than the two marker-gene-based methods MetaPhlAn and mOTU. Filtering out the 1% least abundance species-which were not reliably predicted-helped increase the performance of most profilers by increasing precision but at the cost of recall. However, the use of adaptive filtering thresholds determined from the sample's Shannon index increased the performance of most kmer-based profilers while mitigating the tradeoff between precision and recall. CONCLUSIONS: kmer-based metagenomic pipelines using Kraken/Bracken or CLARK-S performed most robustly across a large variety of microbiome datasets. Removing non-reliably predicted low-abundance species by using diversity-dependent adaptive filtering thresholds further enhanced the performance of these tools. This work demonstrates the applicability of computational pipelines for accurately determining taxonomic profiles in clinical and environmental contexts and exemplifies the power of crowdsourcing for unbiased evaluation.

Comparing the preclinical risk profile of inhalable candidate and potential candidate modified risk tobacco products: A bridging use case.

Cigarette smoking causes major preventable diseases, morbidity, and mortality worldwide. Smoking cessation and prevention of smoking initiation are the preferred means for reducing these risks. Less harmful tobacco products, termed modified-risk tobacco products (MRTP), are being developed as a potential alternative for current adult smokers who would otherwise continue smoking. According to a regulatory framework issued by the US Food and Drug Administration, a manufacturer must provide comprehensive scientific evidence that the product significantly reduces harm and the risk of tobacco-related diseases, in order to obtain marketing authorization for a new MRTP. For new tobacco products similar to an already approved predicate product, the FDA has foreseen a simplified procedure for assessing "substantial equivalence". In this article, we present a use case that bridges the nonclinical evidence from previous studies demonstrating the relatively reduced harm potential of two heat-not-burn products based on different tobacco heating principles. The nonclinical evidence was collected along a "causal chain of events leading to disease" (CELSD) to systematically follow the consequences of reduced exposure to toxicants (relative to cigarette smoke) through increasing levels of biological complexity up to disease manifestation in animal models of human disease. This approach leverages the principles of systems biology and toxicology as a basis for further extrapolation to human studies. The experimental results demonstrate a similarly reduced impact of both products on apical and molecular endpoints, no novel effects not seen with cigarette smoke exposure, and an effect of switching from cigarettes to either MRTP that is comparable to that of complete smoking cessation. Ideally, a subset of representative assays from the presented sequence along the CELSD could be sufficient for predicting similarity or substantial equivalence in the nonclinical impact of novel products; this would require further validation, for which the present use case could serve as a starting point.

Reduced Chronic Toxicity and Carcinogenicity in A/J Mice in Response to Life-Time Exposure to Aerosol From a Heated Tobacco Product Compared With Cigarette Smoke.

We conducted an inhalation study, in accordance with Organisation for Economic Co-operation and Development Test Guideline 453, exposing A/J mice to tobacco heating system (THS) 2.2 aerosol or 3R4F reference cigarette smoke (CS) for up to 18 months to evaluate chronic toxicity and carcinogenicity. All exposed mice showed lower thymus and spleen weight, blood lymphocyte counts, and serum lipid concentrations than sham mice, most likely because of stress and/or nicotine effects. Unlike THS 2.2 aerosol-exposed mice, CS-exposed mice showed increased heart weight, changes in red blood cell profiles and serum liver function parameters. Similarly, increased pulmonary inflammation, altered lung function, and emphysematous changes were observed only in CS-exposed mice. Histopathological changes in other respiratory tract organs were significantly lower in the THS 2.2 aerosol-exposed groups than in the CS-exposed group. Chronic exposure to THS 2.2 aerosol also did not increase the incidence or multiplicity of bronchioloalveolar adenomas or carcinomas relative to sham, whereas CS exposure did. Male THS 2.2 aerosol-exposed mice had a lower survival rate than sham mice, related to an increased incidence of urogenital issues that appears to be related to congenital factors rather than test item exposure. The lower impact of THS 2.2 aerosol exposure on tumor development and chronic toxicity is consistent with the significantly reduced levels of harmful and potentially harmful constituents in THS 2.2 aerosol relative to CS. The totality of the evidence from this study further supports the risk reduction potential of THS 2.2 for lung diseases in comparison with cigarettes.

Toxicological assessment of Tobacco Heating System 2.2: Findings from an independent peer review.

Offering safer alternatives to cigarettes, such as e-cigarettes and heated tobacco products, to smokers who are not willing to quit could reduce the harm caused by smoking. Extensive and rigorous scientific studies are conducted to assess the relative risk of such potentially modified risk tobacco products compared with that of smoking cigarettes. In addition to the peer review of publications reporting individual studies, we aimed to gauge the plausibility of the evidence to the scientific community and appreciate likely necessary additions prior to regulatory submission. Therefore, we sponsored a two-tier peer review organized by an independent third party who identified, recruited, and managed 7 panels of 5-12 experts whose identity remains unknown to us. The reviewers had access to all publications and raw data from preclinical and clinical studies via a web portal. The reviewers were asked questions regarding study design, methods, quality of data, and interpretation of results to judge the validity of the conclusions regarding the relative effects of the Tobacco Heating System 2.2 compared with cigarettes. Once their conclusions were submitted, the experts had the opportunity to participate in an anonymized online debate with their fellow panel members. We present here the results obtained from this innovative peer review effort which revealed supportive or very supportive of the study methods and results, and support the robustness of the studies and validity of the conclusions.

Embracing Transparency Through Data Sharing.

Low rates of reproducibility and translatability of data from nonclinical research have been reported. Major causes of irreproducibility include oversights in study design, failure to characterize reagents and protocols, a lack of access to detailed methods and data, and an absence of universally accepted and applied standards and guidelines. Specific areas of concern include uncharacterized antibodies and cell lines, the use of inappropriate sampling and testing protocols, a lack of transparency and access to raw data, and deficiencies in the translatability of findings to the clinic from studies using animal models of disease. All stakeholders-academia, industry, funding agencies, regulators, nonprofit entities, and publishers-are encouraged to play active roles in addressing these challenges by formulating and promoting access to best practices and standard operating procedures and validating data collaboratively at each step of the biomedical research life cycle.

The sbv IMPROVER Systems Toxicology Computational Challenge: Identification of Human and Species-Independent Blood Response Markers as Predictors of Smoking Exposure and Cessation Status.

Cigarette smoking entails chronic exposure to a mixture of harmful chemicals that trigger molecular changes over time, and is known to increase the risk of developing diseases. Risk assessment in the context of 21st century toxicology relies on the elucidation of mechanisms of toxicity and the identification of exposure response markers, usually from high-throughput data, using advanced computational methodologies. The sbv IMPROVER Systems Toxicology computational challenge (Fall 2015-Spring 2016) aimed to evaluate whether robust and sparse (≤40 genes) human (sub-challenge 1, SC1) and species-independent (sub-challenge 2, SC2) exposure response markers (so called gene signatures) could be extracted from human and mouse blood transcriptomics data of current (S), former (FS) and never (NS) smoke-exposed subjects as predictors of smoking and cessation status. Best-performing computational methods were identified by scoring anonymized participants' predictions. Worldwide participation resulted in 12 (SC1) and six (SC2) final submissions qualified for scoring. The results showed that blood gene expression data were informative to predict smoking exposure (i.e. discriminating smoker versus never or former smokers) status in human and across species with a high level of accuracy. By contrast, the prediction of cessation status (i.e. distinguishing FS from NS) remained challenging, as reflected by lower classification performances. Participants successfully developed inductive predictive models and extracted human and species-independent gene signatures, including genes with high consensus across teams. Post-challenge analyses highlighted "feature selection" as a key step in the process of building a classifier and confirmed the importance of testing a gene signature in independent cohorts to ensure the generalized applicability of a predictive model at a population-based level. In conclusion, the Systems Toxicology challenge demonstrated the feasibility of extracting a consistent blood-based smoke exposure response gene signature and further stressed the importance of independent and unbiased data and method evaluations to provide confidence in systems toxicology-based scientific conclusions.

Interrogating the microbiome: experimental and computational considerations in support of study reproducibility.

The microbiome is an important factor in human health and disease and is investigated to develop novel therapeutics. Metagenomics leverages advances in sequencing technologies and computational analysis to identify and quantify the microorganisms present in a sample. This field has, however, not yet reached maturity and the international metagenomics community, aware of the current limitations and of the necessity for standardization, has started investigating sources of variability in experimental and computational workflows. The first studies have already resulted in the identification of crucial steps and factors affecting metagenomics data quality, quantification and interpretation. This review summarizes experimental and computational considerations for interrogating the microbiome and establishing reproducible and robust analysis workflows.

Supporting evidence-based analysis for modified risk tobacco products through a toxicology data-sharing infrastructure.

The US FDA defines modified risk tobacco products (MRTPs) as products that aim to reduce harm or the risk of tobacco-related disease associated with commercially marketed tobacco products.  Establishing a product's potential as an MRTP requires scientific substantiation including toxicity studies and measures of disease risk relative to those of cigarette smoking.  Best practices encourage verification of the data from such studies through sharing and open standards. Building on the experience gained from the OpenTox project, a proof-of-concept database and website ( INTERVALS) has been developed to share results from both in vivo inhalation studies and in vitro studies conducted by Philip Morris International R&D to assess candidate MRTPs. As datasets are often generated by diverse methods and standards, they need to be traceable, curated, and the methods used well described so that knowledge can be gained using data science principles and tools. The data-management framework described here accounts for the latest standards of data sharing and research reproducibility. Curated data and methods descriptions have been prepared in ISA-Tab format and stored in a database accessible via a search portal on the INTERVALS website. The portal allows users to browse the data by study or mechanism (e.g., inflammation, oxidative stress) and obtain information relevant to study design, methods, and the most important results. Given the successful development of the initial infrastructure, the goal is to grow this initiative and establish a public repository for 21 st-century preclinical systems toxicology MRTP assessment data and results that supports open data principles.

Aerosol from Tobacco Heating System 2.2 has reduced impact on mouse heart gene expression compared with cigarette smoke.

Experimental studies clearly demonstrate a causal effect of cigarette smoking on cardiovascular disease. To reduce the individual risk and population harm caused by smoking, alternative products to cigarettes are being developed. We recently reported on an apolipoprotein E-deficient (Apoe-/-) mouse inhalation study that compared the effects of exposure to aerosol from a candidate modified risk tobacco product, Tobacco Heating System 2.2 (THS2.2), and smoke from the reference cigarette (3R4F) on pulmonary and vascular biology. Here, we applied a transcriptomics approach to evaluate the impact of the exposure to 3R4F smoke and THS2.2 aerosol on heart tissues from the same cohort of mice. The systems response profiles demonstrated that 3R4F smoke exposure led to time-dependent transcriptomics changes (False Discovery Rate (FDR) < 0.05; 44 differentially expressed genes at 3-months; 491 at 8-months). Analysis of differentially expressed genes in the heart tissue indicated that 3R4F exposure induced the downregulation of genes involved in cytoskeleton organization and the contractile function of the heart, notably genes that encode beta actin (Actb), actinin alpha 4 (Actn4), and filamin C (Flnc). This was accompanied by the downregulation of genes related to the inflammatory response. None of these effects were observed in the group exposed to THS2.2 aerosol.

Crowd-Sourced Verification of Computational Methods and Data in Systems Toxicology: A Case Study with a Heat-Not-Burn Candidate Modified Risk Tobacco Product.

Systems toxicology intends to quantify the effect of toxic molecules in biological systems and unravel their mechanisms of toxicity. The development of advanced computational methods is required for analyzing and integrating high throughput data generated for this purpose as well as for extrapolating predictive toxicological outcomes and risk estimates. To ensure the performance and reliability of the methods and verify conclusions from systems toxicology data analysis, it is important to conduct unbiased evaluations by independent third parties. As a case study, we report here the results of an independent verification of methods and data in systems toxicology by crowdsourcing. The sbv IMPROVER systems toxicology computational challenge aimed to evaluate computational methods for the development of blood-based gene expression signature classification models with the ability to predict smoking exposure status. Participants created/trained models on blood gene expression data sets including smokers/mice exposed to 3R4F (a reference cigarette) or noncurrent smokers/Sham (mice exposed to air). Participants applied their models on unseen data to predict whether subjects classify closer to smoke-exposed or nonsmoke exposed groups. The data sets also included data from subjects that had been exposed to potential modified risk tobacco products (MRTPs) or that had switched to a MRTP after exposure to conventional cigarette smoke. The scoring of anonymized participants' predictions was done using predefined metrics. The top 3 performers' methods predicted class labels with area under the precision recall scores above 0.9. Furthermore, although various computational approaches were used, the crowd's results confirmed our own data analysis outcomes with regards to the classification of MRTP-related samples. Mice exposed directly to a MRTP were classified closer to the Sham group. After switching to a MRTP, the confidence that subjects belonged to the smoke-exposed group decreased significantly. Smoking exposure gene signatures that contributed to the group separation included a core set of genes highly consistent across teams such as AHRR, LRRN3, SASH1, and P2RY6. In conclusion, crowdsourcing constitutes a pertinent approach, in complement to the classical peer review process, to independently and unbiasedly verify computational methods and data for risk assessment using systems toxicology.

Evaluation of the Tobacco Heating System 2.2. Part 4: 90-day OECD 413 rat inhalation study with systems toxicology endpoints demonstrates reduced exposure effects compared with cigarette smoke.

The objective of the study was to characterize the toxicity from sub-chronic inhalation of test atmospheres from the candidate modified risk tobacco product (MRTP), Tobacco Heating System version 2.2 (THS2.2), and to compare it with that of the 3R4F reference cigarette. A 90-day nose-only inhalation study on Sprague-Dawley rats was performed, combining classical and systems toxicology approaches. Reduction in respiratory minute volume, degree of lung inflammation, and histopathological findings in the respiratory tract organs were significantly less pronounced in THS2.2-exposed groups compared with 3R4F-exposed groups. Transcriptomics data obtained from nasal epithelium and lung parenchyma showed concentration-dependent differential gene expression following 3R4F exposure that was less pronounced in the THS2.2-exposed groups. Molecular network analysis showed that inflammatory processes were the most affected by 3R4F, while the extent of THS2.2 impact was much lower. Most other toxicological endpoints evaluated did not show exposure-related effects. Where findings were observed, the effects were similar in 3R4F- and THS2.2-exposed animals. In summary, toxicological changes observed in the respiratory tract organs of THS2.2 aerosol-exposed rats were much less pronounced than in 3R4F-exposed rats while other toxicological endpoints either showed no exposure-related effects or were comparable to what was observed in the 3R4F-exposed rats.

Alterations in Serum Polyunsaturated Fatty Acids and Eicosanoids in Patients with Mild to Moderate Chronic Obstructive Pulmonary Disease (COPD).

Smoking is a major risk factor for several diseases including chronic obstructive pulmonary disease (COPD). To better understand the systemic effects of cigarette smoke exposure and mild to moderate COPD-and to support future biomarker development-we profiled the serum lipidomes of healthy smokers, smokers with mild to moderate COPD (GOLD stages 1 and 2), former smokers, and never-smokers (n = 40 per group) (ClinicalTrials.gov registration: NCT01780298). Serum lipidome profiling was conducted with untargeted and targeted mass spectrometry-based lipidomics. Guided by weighted lipid co-expression network analysis, we identified three main trends comparing smokers, especially those with COPD, with non-smokers: a general increase in glycero(phospho)lipids, including triglycerols; changes in fatty acid desaturation (decrease in ω-3 polyunsaturated fatty acids, and an increase in monounsaturated fatty acids); and an imbalance in eicosanoids (increase in 11,12- and 14,15-DHETs (dihydroxyeicosatrienoic acids), and a decrease in 9- and 13-HODEs (hydroxyoctadecadienoic acids)). The lipidome profiles supported classification of study subjects as smokers or non-smokers, but were not sufficient to distinguish between smokers with and without COPD. Overall, our study yielded further insights into the complex interplay between smoke exposure, lung disease, and systemic alterations in serum lipid profiles.

Crowdsourcing and curation: perspectives from biology and natural language processing.

Crowdsourcing is increasingly utilized for performing tasks in both natural language processing and biocuration. Although there have been many applications of crowdsourcing in these fields, there have been fewer high-level discussions of the methodology and its applicability to biocuration. This paper explores crowdsourcing for biocuration through several case studies that highlight different ways of leveraging 'the crowd'; these raise issues about the kind(s) of expertise needed, the motivations of participants, and questions related to feasibility, cost and quality. The paper is an outgrowth of a panel session held at BioCreative V (Seville, September 9-11, 2015). The session consisted of four short talks, followed by a discussion. In their talks, the panelists explored the role of expertise and the potential to improve crowd performance by training; the challenge of decomposing tasks to make them amenable to crowdsourcing; and the capture of biological data and metadata through community editing.Database URL: http://www.mitre.org/publications/technical-papers/crowdsourcing-and-curation-perspectives.

Community-Reviewed Biological Network Models for Toxicology and Drug Discovery Applications.

Biological network models offer a framework for understanding disease by describing the relationships between the mechanisms involved in the regulation of biological processes. Crowdsourcing can efficiently gather feedback from a wide audience with varying expertise. In the Network Verification Challenge, scientists verified and enhanced a set of 46 biological networks relevant to lung and chronic obstructive pulmonary disease. The networks were built using Biological Expression Language and contain detailed information for each node and edge, including supporting evidence from the literature. Network scoring of public transcriptomics data inferred perturbation of a subset of mechanisms and networks that matched the measured outcomes. These results, based on a computable network approach, can be used to identify novel mechanisms activated in disease, quantitatively compare different treatments and time points, and allow for assessment of data with low signal. These networks are periodically verified by the crowd to maintain an up-to-date suite of networks for toxicology and drug discovery applications.

The Apoe(-/-) mouse model: a suitable model to study cardiovascular and respiratory diseases in the context of cigarette smoke exposure and harm reduction.

Atherosclerosis-prone apolipoprotein E-deficient (Apoe(-/-)) mice display poor lipoprotein clearance with subsequent accumulation of cholesterol ester-enriched particles in the blood, which promote the development of atherosclerotic plaques. Therefore, the Apoe(-/-) mouse model is well established for the study of human atherosclerosis. The systemic proinflammatory status of Apoe(-/-) mice also makes them good candidates for studying chronic obstructive pulmonary disease, characterized by pulmonary inflammation, airway obstruction, and emphysema, and which shares several risk factors with cardiovascular diseases, including smoking. Herein, we review the results from published studies using Apoe(-/-) mice, with a particular focus on work conducted in the context of cigarette smoke inhalation studies. The findings from these studies highlight the suitability of this animal model for researching the effects of cigarette smoking on atherosclerosis and emphysema.

Effects of cigarette smoke, cessation and switching to a candidate modified risk tobacco product on the liver in Apoe -/- mice--a systems toxicology analysis.

The liver is one of the most important organs involved in elimination of xenobiotic and potentially toxic substances. Cigarette smoke (CS) contains more than 7000 chemicals, including those that exert biological effects and cause smoking-related diseases. Though CS is not directly hepatotoxic, a growing body of evidence suggests that it may exacerbate pre-existing chronic liver disease. In this study, we integrated toxicological endpoints with molecular measurements and computational analyses to investigate effects of exposures on the livers of Apoe(-/- )mice. Mice were exposed to 3R4F reference CS, to an aerosol from the Tobacco Heating System (THS) 2.2, a candidate modified risk tobacco product (MRTP) or to filtered air (Sham) for up to 8 months. THS2.2 takes advantage of a "heat-not-burn" technology that, by heating tobacco, avoids pyrogenesis and pyrosynthesis. After CS exposure for 2 months, some groups were either switched to the MRTP or filtered air. While no group showed clear signs of hepatotoxicity, integrative analysis of proteomics and transcriptomics data showed a CS-dependent impairment of specific biological networks. These networks included lipid and xenobiotic metabolism and iron homeostasis that likely contributed synergistically to exacerbating oxidative stress. In contrast, most proteomic and transcriptomic changes were lower in mice exposed to THS2.2 and in the cessation and switching groups compared to the CS group. Our findings elucidate the complex biological responses of the liver to CS exposure. Furthermore, they provide evidence that THS2.2 aerosol has reduced biological effects, as compared with CS, on the livers of Apoe(-/- )mice.

SETD7 Regulates the Differentiation of Human Embryonic Stem Cells.

The successful use of specialized cells in regenerative medicine requires an optimization in the differentiation protocols that are currently used. Understanding the molecular events that take place during the differentiation of human pluripotent cells is essential for the improvement of these protocols and the generation of high quality differentiated cells. In an effort to understand the molecular mechanisms that govern differentiation we identify the methyltransferase SETD7 as highly induced during the differentiation of human embryonic stem cells and differentially expressed between induced pluripotent cells and somatic cells. Knock-down of SETD7 causes differentiation defects in human embryonic stem cell including delay in both the silencing of pluripotency-related genes and the induction of differentiation genes. We show that SETD7 methylates linker histone H1 in vitro causing conformational changes in H1. These effects correlate with a decrease in the recruitment of H1 to the pluripotency genes OCT4 and NANOG during differentiation in the SETD7 knock down that might affect the proper silencing of these genes during differentiation.

Comprehensive systems biology analysis of a 7-month cigarette smoke inhalation study in C57BL/6 mice.

Smoking of combustible cigarettes has a major impact on human health. Using a systems toxicology approach in a model of chronic obstructive pulmonary disease (C57BL/6 mice), we assessed the health consequences in mice of an aerosol derived from a prototype modified risk tobacco product (pMRTP) as compared to conventional cigarettes. We investigated physiological and histological endpoints in parallel with transcriptomics, lipidomics, and proteomics profiles in mice exposed to a reference cigarette (3R4F) smoke or a pMRTP aerosol for up to 7 months. We also included a cessation group and a switching-to-pMRTP group (after 2 months of 3R4F exposure) in addition to the control (fresh air-exposed) group, to understand the potential risk reduction of switching to pMRTP compared with continuous 3R4F exposure and cessation. The present manuscript describes the study design, setup, and implementation, as well as the generation, processing, and quality control analysis of the toxicology and 'omics' datasets that are accessible in public repositories for further analyses.