Variability and bias in microbiome metagenomic sequencing: an interlaboratory study comparing experimental protocols.

Several studies have documented the significant impact of methodological choices in microbiome analyses. The myriad of methodological options available complicate the replication of results and generally limit the comparability of findings between independent studies that use differing techniques and measurement pipelines. Here we describe the Mosaic Standards Challenge (MSC), an international interlaboratory study designed to assess the impact of methodological variables on the results. The MSC did not prescribe methods but rather asked participating labs to analyze 7 shared reference samples (5 × human stool samples and 2 × mock communities) using their standard laboratory methods. To capture the array of methodological variables, each participating lab completed a metadata reporting sheet that included 100 different questions regarding the details of their protocol. The goal of this study was to survey the methodological landscape for microbiome metagenomic sequencing (MGS) analyses and the impact of methodological decisions on metagenomic sequencing results. A total of 44 labs participated in the MSC by submitting results (16S or WGS) along with accompanying metadata; thirty 16S rRNA gene amplicon datasets and 14 WGS datasets were collected. The inclusion of two types of reference materials (human stool and mock communities) enabled analysis of both MGS measurement variability between different protocols using the biologically-relevant stool samples, and MGS bias with respect to ground truth values using the DNA mixtures. Owing to the compositional nature of MGS measurements, analyses were conducted on the ratio of Firmicutes: Bacteroidetes allowing us to directly apply common statistical methods. The resulting analysis demonstrated that protocol choices have significant effects, including both bias of the MGS measurement associated with a particular methodological choices, as well as effects on measurement robustness as observed through the spread of results between labs making similar methodological choices. In the analysis of the DNA mock communities, MGS measurement bias was observed even when there was general consensus among the participating laboratories. This study was the result of a collaborative effort that included academic, commercial, and government labs. In addition to highlighting the impact of different methodological decisions on MGS result comparability, this work also provides insights for consideration in future microbiome measurement study design.

Benchmarking of virome metagenomic analysis approaches using a large, 60+ members, viral synthetic community.

IMPORTANCE: We report here efforts to benchmark performance of two widespread approaches for virome analysis, which target either virion-associated nucleic acids (VANA) or highly purified double-stranded RNAs (dsRNAs). This was achieved using synthetic communities of varying complexity levels, up to a highly complex community of 72 viral agents (115 viral molecules) comprising isolates from 21 families and 61 genera of plant viruses. The results obtained confirm that the dsRNA-based approach provides a more complete representation of the RNA virome, in particular, for high complexity ones. However, for viromes of low to medium complexity, VANA appears a reasonable alternative and would be the preferred choice if analysis of DNA viruses is of importance. Several parameters impacting performance were identified as well as a direct relationship between the completeness of virome description and sample sequencing depth. The strategy, results, and tools used here should prove useful in a range of virome analysis efforts.

Multicenter assessment of shotgun metagenomics for pathogen detection.

BACKGROUND: Shotgun metagenomics has been used clinically for diagnosing infectious diseases. However, most technical assessments have been limited to individual sets of reference standards, experimental workflows, and laboratories. METHODS: A reference panel and performance metrics were designed and used to examine the performance of shotgun metagenomics at 17 laboratories in a coordinated collaborative study. We comprehensively assessed the reliability, key performance determinants, reproducibility, and quantitative potential. FINDINGS: Assay performance varied significantly across sites and microbial classes, with a read depth of 20 millions as a generally cost-efficient assay setting. Results of mapped reads by shotgun metagenomics could indicate relative and intra-site (but not absolute or inter-site) microbial abundance. INTERPRETATION: Assay performance was significantly impacted by the microbial type, the host context, and read depth, which emphasizes the importance of these factors when designing reference reagents and benchmarking studies. Across sites, workflows and platforms, false positive reporting and considerable site/library effects were common challenges to the assay's accuracy and quantifiability. Our study also suggested that laboratory-developed shotgun metagenomics tests for pathogen detection should aim to detect microbes at 500 CFU/mL (or copies/mL) in a clinically relevant host context (10^5 human cells/mL) within a 24h turn-around time, and with an efficient read depth of 20M. FUNDING: This work was supported by National Science and Technology Major Project of China (2018ZX10102001).

Benchmarking laboratory processes to characterise low-biomass respiratory microbiota.

The low biomass of respiratory samples makes it difficult to accurately characterise the microbial community composition. PCR conditions and contaminating microbial DNA can alter the biological profile. The objective of this study was to benchmark the currently available laboratory protocols to accurately analyse the microbial community of low biomass samples. To study the effect of PCR conditions on the microbial community profile, we amplified the 16S rRNA gene of respiratory samples using various bacterial loads and different number of PCR cycles. Libraries were purified by gel electrophoresis or AMPure XP and sequenced by V2 or V3 MiSeq reagent kits by Illumina sequencing. The positive control was diluted in different solvents. PCR conditions had no significant influence on the microbial community profile of low biomass samples. Purification methods and MiSeq reagent kits provided nearly similar microbiota profiles (paired Bray-Curtis dissimilarity median: 0.03 and 0.05, respectively). While profiles of positive controls were significantly influenced by the type of dilution solvent, the theoretical profile of the Zymo mock was most accurately analysed when the Zymo mock was diluted in elution buffer (difference compared to the theoretical Zymo mock: 21.6% for elution buffer, 29.2% for Milli-Q, and 79.6% for DNA/RNA shield). Microbiota profiles of DNA blanks formed a distinct cluster compared to low biomass samples, demonstrating that low biomass samples can accurately be distinguished from DNA blanks. In summary, to accurately characterise the microbial community composition we recommend 1. amplification of the obtained microbial DNA with 30 PCR cycles, 2. purifying amplicon pools by two consecutive AMPure XP steps and 3. sequence the pooled amplicons by V3 MiSeq reagent kit. The benchmarked standardized laboratory workflow presented here ensures comparability of results within and between low biomass microbiome studies.

Comparison of targeted metagenomics and IS-Pro methods for analysing the lung microbiome.

BACKGROUND: Targeted metagenomics and IS-Pro method are two of the many methods that have been used to study the microbiome. The two methods target different regions of the 16 S rRNA gene. The aim of this study was to compare targeted metagenomics and IS-Pro methods for the ability to discern the microbial composition of the lung microbiome of COPD patients. METHODS: Spontaneously expectorated sputum specimens were collected from COPD patients. Bacterial DNA was extracted and used for targeted metagenomics and IS-Pro method. The analysis was performed using QIIME2 (targeted metagenomics) and IS-Pro software (IS-Pro method). Additionally, a laboratory cost per isolate and time analysis was performed for each method. RESULTS: Statistically significant differences were observed in alpha diversity when targeted metagenomics and IS-Pro methods' data were compared using the Shannon diversity measure (p-value = 0.0006) but not with the Simpson diversity measure (p-value = 0.84). Distinct clusters with no overlap between the two technologies were observed for beta diversity. Targeted metagenomics had a lower relative abundance of phyla, such as the Proteobacteria, and higher relative abundance of phyla, such as Firmicutes when compared to the IS-Pro method. Haemophilus, Prevotella and Streptococcus were most prevalent genera across both methods. Targeted metagenomics classified 23 % (144/631) of OTUs to a species level, whereas IS-Pro method classified 86 % (55/64) of OTUs to a species level. However, unclassified OTUs accounted for a higher relative abundance when using the IS-Pro method (35 %) compared to targeted metagenomics (5 %). The two methods performed comparably in terms of cost and time; however, the IS-Pro method was more user-friendly. CONCLUSIONS: It is essential to understand the value of different methods for characterisation of the microbiome. Targeted metagenomics and IS-Pro methods showed differences in ability in identifying and characterising OTUs, diversity and microbial composition of the lung microbiome. The IS-Pro method might miss relevant species and could inflate the abundance of Proteobacteria. However, the IS-Pro kit identified most of the important lung pathogens, such as Burkholderia and Pseudomonas and may work in a more diagnostics-orientated setting. Both methods were comparable in terms of cost and time; however, the IS-Pro method was easier to use.

Detection of cryptogenic malignancies from metagenomic whole genome sequencing of body fluids.

BACKGROUND: Metagenomic next-generation sequencing (mNGS) of body fluids is an emerging approach to identify occult pathogens in undiagnosed patients. We hypothesized that metagenomic testing can be simultaneously used to detect malignant neoplasms in addition to infectious pathogens. METHODS: From two independent studies (n = 205), we used human data generated from a metagenomic sequencing pipeline to simultaneously screen for malignancies by copy number variation (CNV) detection. In the first case-control study, we analyzed body fluid samples (n = 124) from patients with a clinical diagnosis of either malignancy (positive cases, n = 65) or infection (negative controls, n = 59). In a second verification cohort, we analyzed a series of consecutive cases (n = 81) sent to cytology for malignancy workup that included malignant positives (n = 32), negatives (n = 18), or cases with an unclear gold standard (n = 31). RESULTS: The overall CNV test sensitivity across all studies was 87% (55 of 63) in patients with malignancies confirmed by conventional cytology and/or flow cytometry testing and 68% (23 of 34) in patients who were ultimately diagnosed with cancer but negative by conventional testing. Specificity was 100% (95% CI 95-100%) with no false positives detected in 77 negative controls. In one example, a patient hospitalized with an unknown pulmonary illness had non-diagnostic lung biopsies, while CNVs implicating a malignancy were detectable from bronchoalveolar fluid. CONCLUSIONS: Metagenomic sequencing of body fluids can be used to identify undetected malignant neoplasms through copy number variation detection. This study illustrates the potential clinical utility of a single metagenomic test to uncover the cause of undiagnosed acute illnesses due to cancer or infection using the same specimen.

Optimising sampling and analysis protocols in environmental DNA studies.

Ecological surveys risk incurring false negative and false positive detections of the target species. With indirect survey methods, such as environmental DNA, such error can occur at two stages: sample collection and laboratory analysis. Here we analyse a large qPCR based eDNA data set using two occupancy models, one of which accounts for false positive error by Griffin et al. (J R Stat Soc Ser C Appl Stat 69: 377-392, 2020), and a second that assumes no false positive error by Stratton et al. (Methods Ecol Evol 11: 1113-1120, 2020). Additionally, we apply the Griffin et al. (2020) model to simulated data to determine optimal levels of replication at both sampling stages. The Stratton et al. (2020) model, which assumes no false positive results, consistently overestimated both overall and individual site occupancy compared to both the Griffin et al. (2020) model and to previous estimates of pond occupancy for the target species. The inclusion of replication at both stages of eDNA analysis (sample collection and in the laboratory) reduces both bias and credible interval width in estimates of both occupancy and detectability. Even the collection of > 1 sample from a site can improve parameter estimates more than having a high number of replicates only within the laboratory analysis.

How accurately can we assess zoonotic risk?

Identifying the animal reservoirs from which zoonotic viruses will likely emerge is central to understanding the determinants of disease emergence. Accordingly, there has been an increase in studies attempting zoonotic "risk assessment." Herein, we demonstrate that the virological data on which these analyses are conducted are incomplete, biased, and rapidly changing with ongoing virus discovery. Together, these shortcomings suggest that attempts to assess zoonotic risk using available virological data are likely to be inaccurate and largely only identify those host taxa that have been studied most extensively. We suggest that virus surveillance at the human-animal interface may be more productive.

Accurate and sensitive detection of microbial eukaryotes from whole metagenome shotgun sequencing.

BACKGROUND: Microbial eukaryotes are found alongside bacteria and archaea in natural microbial systems, including host-associated microbiomes. While microbial eukaryotes are critical to these communities, they are challenging to study with shotgun sequencing techniques and are therefore often excluded. RESULTS: Here, we present EukDetect, a bioinformatics method to identify eukaryotes in shotgun metagenomic sequencing data. Our approach uses a database of 521,824 universal marker genes from 241 conserved gene families, which we curated from 3713 fungal, protist, non-vertebrate metazoan, and non-streptophyte archaeplastida genomes and transcriptomes. EukDetect has a broad taxonomic coverage of microbial eukaryotes, performs well on low-abundance and closely related species, and is resilient against bacterial contamination in eukaryotic genomes. Using EukDetect, we describe the spatial distribution of eukaryotes along the human gastrointestinal tract, showing that fungi and protists are present in the lumen and mucosa throughout the large intestine. We discover that there is a succession of eukaryotes that colonize the human gut during the first years of life, mirroring patterns of developmental succession observed in gut bacteria. By comparing DNA and RNA sequencing of paired samples from human stool, we find that many eukaryotes continue active transcription after passage through the gut, though some do not, suggesting they are dormant or nonviable. We analyze metagenomic data from the Baltic Sea and find that eukaryotes differ across locations and salinity gradients. Finally, we observe eukaryotes in Arabidopsis leaf samples, many of which are not identifiable from public protein databases. CONCLUSIONS: EukDetect provides an automated and reliable way to characterize eukaryotes in shotgun sequencing datasets from diverse microbiomes. We demonstrate that it enables discoveries that would be missed or clouded by false positives with standard shotgun sequence analysis. EukDetect will greatly advance our understanding of how microbial eukaryotes contribute to microbiomes. Video abstract.

Kssd: sequence dimensionality reduction by k-mer substring space sampling enables real-time large-scale datasets analysis.

Here, we develop k -mer substring space decomposition (Kssd), a sketching technique which is significantly faster and more accurate than current sketching methods. We show that it is the only method that can be used for large-scale dataset comparisons at population resolution on simulated and real data. Using Kssd, we prioritize references for all 1,019,179 bacteria whole genome sequencing (WGS) runs from NCBI Sequence Read Archive and find misidentification or contamination in 6164 of these. Additionally, we analyze WGS and exome runs of samples from the 1000 Genomes Project.

Current challenges and best-practice protocols for microbiome analysis.

Analyzing the microbiome of diverse species and environments using next-generation sequencing techniques has significantly enhanced our understanding on metabolic, physiological and ecological roles of environmental microorganisms. However, the analysis of the microbiome is affected by experimental conditions (e.g. sequencing errors and genomic repeats) and computationally intensive and cumbersome downstream analysis (e.g. quality control, assembly, binning and statistical analyses). Moreover, the introduction of new sequencing technologies and protocols led to a flood of new methodologies, which also have an immediate effect on the results of the analyses. The aim of this work is to review the most important workflows for 16S rRNA sequencing and shotgun and long-read metagenomics, as well as to provide best-practice protocols on experimental design, sample processing, sequencing, assembly, binning, annotation and visualization. To simplify and standardize the computational analysis, we provide a set of best-practice workflows for 16S rRNA and metagenomic sequencing data (available at https://github.com/grimmlab/MicrobiomeBestPracticeReview).

Meta-Prism: Ultra-fast and highly accurate microbial community structure search utilizing dual indexing and parallel computation.

Microbiome samples are accumulating at an unprecedented speed. As a result, a massive amount of samples have become available for the mining of the intrinsic patterns among them. However, due to the lack of advanced computational tools, fast yet accurate comparisons and searches among thousands to millions of samples are still in urgent need. In this work, we proposed the Meta-Prism method for comparing and searching the microbial community structures amongst tens of thousands of samples. Meta-Prism is at least 10 times faster than contemporary methods serving the same purpose and can provide very accurate search results. The method is based on three computational techniques: dual-indexing approach for sample subgrouping, refined scoring function that could scrutinize the minute differences among samples, and parallel computation on CPU or GPU. The superiority of Meta-Prism on speed and accuracy for multiple sample searches is proven based on searching against ten thousand samples derived from both human and environments. Therefore, Meta-Prism could facilitate similarity search and in-depth understanding among massive number of heterogenous samples in the microbiome universe. The codes of Meta-Prism are available at: https://github.com/HUST-NingKang-Lab/metaPrism.

Comparison of microbiome samples: methods and computational challenges.

The study of microbial communities crucially relies on the comparison of metagenomic next-generation sequencing data sets, for which several methods have been designed in recent years. Here, we review three key challenges in the comparison of such data sets: species identification and quantification, the efficient computation of distances between metagenomic samples and the identification of metagenomic features associated with a phenotype such as disease status. We present current solutions for such challenges, considering both reference-based methods relying on a database of reference genomes and reference-free methods working directly on all sequencing reads from the samples.

16S rRNA Gene Copy Number Normalization Does Not Provide More Reliable Conclusions in Metataxonomic Surveys.

Sequencing 16S rRNA gene amplicons is the gold standard to uncover the composition of prokaryotic communities. The presence of multiple copies of this gene makes the community abundance data distorted and gene copy normalization (GCN) necessary for correction. Even though GCN of 16S data provided a picture closer to the metagenome before, it should also be compared with communities of known composition due to the fact that library preparation is prone to methodological biases. Here, we process 16S rRNA gene amplicon data from eleven simple mock communities with DADA2 and estimate the impact of GCN. In all cases, the mock community composition derived from the 16S sequencing differs from those expected, and GCN fails to improve the classification for most of the analysed communities. Our approach provides empirical evidence that GCN does not improve the 16S target sequencing analyses in real scenarios. We therefore question the use of GCN for metataxonomic surveys until a more comprehensive catalogue of copy numbers becomes available.

Culture-free genome-wide locus sequence typing (GLST) provides new perspectives on Trypanosoma cruzi dispersal and infection complexity.

Analysis of genetic polymorphism is a powerful tool for epidemiological surveillance and research. Powerful inference from pathogen genetic variation, however, is often restrained by limited access to representative target DNA, especially in the study of obligate parasitic species for which ex vivo culture is resource-intensive or bias-prone. Modern sequence capture methods enable pathogen genetic variation to be analyzed directly from host/vector material but are often too complex and expensive for resource-poor settings where infectious diseases prevail. This study proposes a simple, cost-effective 'genome-wide locus sequence typing' (GLST) tool based on massive parallel amplification of information hotspots throughout the target pathogen genome. The multiplexed polymerase chain reaction amplifies hundreds of different, user-defined genetic targets in a single reaction tube, and subsequent agarose gel-based clean-up and barcoding completes library preparation at under 4 USD per sample. Our study generates a flexible GLST primer panel design workflow for Trypanosoma cruzi, the parasitic agent of Chagas disease. We successfully apply our 203-target GLST panel to direct, culture-free metagenomic extracts from triatomine vectors containing a minimum of 3.69 pg/µl T. cruzi DNA and further elaborate on method performance by sequencing GLST libraries from T. cruzi reference clones representing discrete typing units (DTUs) TcI, TcIII, TcIV, TcV and TcVI. The 780 SNP sites we identify in the sample set repeatably distinguish parasites infecting sympatric vectors and detect correlations between genetic and geographic distances at regional (< 150 km) as well as continental scales. The markers also clearly separate TcI, TcIII, TcIV and TcV + TcVI and appear to distinguish multiclonal infections within TcI. We discuss the advantages, limitations and prospects of our method across a spectrum of epidemiological research.

Metagenomic next generation sequencing for the diagnosis of tuberculosis meningitis: A systematic review and meta-analysis.

BACKGROUND: Tuberculous meningitis (TBM) is a severe form of extrapulmonary tuberculosis and its early diagnosis is very difficult leading to present with severe disability or die. The current study aimed to assess the accuracy of metagenomic next generation sequencing (mNGS) for TBM, and to identify a new test for the early diagnosis of TBM. METHODS: We searched for articles published in Embase, PubMed, Cochrane Library, China National Knowledge Infrastructure, and Wanfang Data up to June 30, 2020 for studies that assessed the efficacy of mNGS for the diagnosis of TBM. Then, the accuracy between mNGS and a composite reference standard (CRS) in these articles was compared using the meta-analysis approach. RESULTS: Four independent studies with 342 samples comparing mNGS and a CRS were included in this study. The sensitivity of mNGS for TBM diagnosis ranged from 27% to 84%. The combined sensitivity of mNGS was 61%, and the I2 value was 92%. Moreover, the specificity of mNGS for TBM diagnosis ranged from 96% to 100%. The combined specificity of mNGS was 98%, and the I2 value was 74%. The heterogeneity between studies in terms of sensitivity and specificity was significant. The area under the curve (AUC) of the summary receiver operating characteristic curve (SROC) of mNGS for TBM was 0.98. CONCLUSIONS: The sensitivity of mNGS for TBM diagnosis was moderate. Furthermore, the specificity was extremely high, and the AUC of the SROC indicated a very good diagnostic efficacy. mNGS could be used as an early diagnostic method for TBM, however, the results should be treated with caution for the heterogeneity between studies was extremely significant. SYSTEMATIC REVIEW REGISTRATION: INPLASY202070100.

Metagenome Proteins and Database Contamination.

Continued influx of metagenome-derived proteins with misannotated taxonomy into conventional databases, including RefSeq, threatens to eliminate the value of taxonomy identifiers. To prevent this, urgent efforts should be undertaken by submitters of metagenomic data sets as well as by database managers.

Navigating Clinical Utilization of Direct-from-Specimen Metagenomic Pathogen Detection: Clinical Applications, Limitations, and Testing Recommendations.

BACKGROUND: Metagenomic next generation sequencing (mNGS) is becoming increasingly available for pathogen detection directly from clinical specimens. These tests use target-independent, shotgun sequencing to detect potentially unlimited organisms. The promise of this methodology to aid infection diagnosis is demonstrated through early case reports and clinical studies. However, the optimal role of mNGS in clinical microbiology remains uncertain. CONTENT: We reviewed studies reporting clinical use of mNGS for pathogen detection from various specimen types, including cerebrospinal fluid, plasma, lower respiratory specimens, and others. Published clinical study data were critically evaluated and summarized to identify promising clinical indications for mNGS-based testing, to assess the clinical impact of mNGS for each indication, and to recognize test limitations. Based on these clinical studies, early testing recommendations are made to guide clinical utilization of mNGS for pathogen detection. Finally, current barriers to routine clinical laboratory implementation of mNGS tests are highlighted. SUMMARY: The promise of direct-from-specimen mNGS to enable challenging infection diagnoses has been demonstrated through early clinical studies of patients with meningitis or encephalitis, invasive fungal infections, community acquired pneumonia, and other clinical indications. However, the proportion of patient cases with positive clinical impact due to mNGS testing is low in published studies and the cost of testing is high, emphasizing the importance of improving our understanding of 'when to test' and for which patients mNGS testing is appropriate.

Do the human gut metagenomic species possess the minimal set of core functionalities necessary for life?

BACKGROUND: Advances in bioinformatics recently allowed for the recovery of 'metagenomes assembled genomes' from human microbiome studies carried on with shotgun sequencing techniques. Such approach is used as a mean to discover new unclassified metagenomic species, putative biological entities having distinct metabolic traits. RESULTS: In the present analysis we compare 400 genomes from isolates available on NCBI database and 10,000 human gut metagenomic species, screening all of them for the presence of a minimal set of core functionalities necessary, but not sufficient, for life. As a result, the metagenome-assembled genomes resulted systematically depleted in genes encoding for essential functions apparently needed to support autonomous bacterial life. CONCLUSIONS: The relevant degree of lacking core functionalities that we observed in metagenome-assembled genomes raises some concerns about the effective completeness of metagenome-assembled genomes, suggesting caution in extrapolating biological information about their metabolic propensity and ecology in a complex environment like the human gastrointestinal tract.