Variability and bias in microbiome metagenomic sequencing: an interlaboratory study comparing experimental protocols.

Several studies have documented the significant impact of methodological choices in microbiome analyses. The myriad of methodological options available complicate the replication of results and generally limit the comparability of findings between independent studies that use differing techniques and measurement pipelines. Here we describe the Mosaic Standards Challenge (MSC), an international interlaboratory study designed to assess the impact of methodological variables on the results. The MSC did not prescribe methods but rather asked participating labs to analyze 7 shared reference samples (5 × human stool samples and 2 × mock communities) using their standard laboratory methods. To capture the array of methodological variables, each participating lab completed a metadata reporting sheet that included 100 different questions regarding the details of their protocol. The goal of this study was to survey the methodological landscape for microbiome metagenomic sequencing (MGS) analyses and the impact of methodological decisions on metagenomic sequencing results. A total of 44 labs participated in the MSC by submitting results (16S or WGS) along with accompanying metadata; thirty 16S rRNA gene amplicon datasets and 14 WGS datasets were collected. The inclusion of two types of reference materials (human stool and mock communities) enabled analysis of both MGS measurement variability between different protocols using the biologically-relevant stool samples, and MGS bias with respect to ground truth values using the DNA mixtures. Owing to the compositional nature of MGS measurements, analyses were conducted on the ratio of Firmicutes: Bacteroidetes allowing us to directly apply common statistical methods. The resulting analysis demonstrated that protocol choices have significant effects, including both bias of the MGS measurement associated with a particular methodological choices, as well as effects on measurement robustness as observed through the spread of results between labs making similar methodological choices. In the analysis of the DNA mock communities, MGS measurement bias was observed even when there was general consensus among the participating laboratories. This study was the result of a collaborative effort that included academic, commercial, and government labs. In addition to highlighting the impact of different methodological decisions on MGS result comparability, this work also provides insights for consideration in future microbiome measurement study design.

Amplicon Sequencing Minimal Information (ASqMI): Quality and Reporting Guidelines for Actionable Calls in Biodefense Applications.

BACKGROUND: Accurate, high-confidence data is critical for assessing potential biothreat incidents. In a biothreat event, false-negative and -positive results have serious consequences. Worst case scenarios can result in unnecessary shutdowns or fatalities at an exorbitant monetary and psychological cost, respectively. Quantitative PCR assays for agents of interest have been successfully used for routine biosurveillance. Recently, there has been increased impetus for adoption of amplicon sequencing (AS) for biosurveillance because it enables discrimination of true positives from near-neighbor false positives, as well as broad, simultaneous detection of many targets in many pathogens in a high-throughput scheme. However, the high sensitivity of AS can lead to false positives. Appropriate controls and workflow reporting can help address these challenges. OBJECTIVES: Data reporting standards are critical to data trustworthiness. The standards presented herein aim to provide a framework for method quality assessment in biodetection. METHODS: We present a set of standards, Amplicon Sequencing Minimal Information (ASqMI), developed under the auspices of the AOAC INTERNATIONAL Stakeholder Program on Agent Detection Assays for making actionable calls in biosurveillance applications. In addition to the first minimum information guidelines for AS, we provide a controls checklist and scoring scheme to assure AS run quality and assess potential sample contamination. RESULTS: Adoption of the ASqMI guidelines will improve data quality, help track workflow performance, and ultimately provide decision makers confidence to trust the results of this new and powerful technology. CONCLUSION: AS workflows can provide robust, confident calls for biodetection; however, due diligence in reporting and controls are needed. The ASqMI guideline is the first AS minimum reporting guidance document that also provides the means for end users to evaluate their workflows to improve confidence. HIGHLIGHTS: Standardized reporting guidance for actionable calls is critical to ensuring trustworthy data.

Interlaboratory performance and quantitative PCR data acceptance metrics for NIST SRM® 2917.

Surface water quality quantitative polymerase chain reaction (qPCR) technologies are expanding from a subject of research to routine environmental and public health laboratory testing. Readily available, reliable reference material is needed to interpret qPCR measurements, particularly across laboratories. Standard Reference Material® 2917 (NIST SRM® 2917) is a DNA plasmid construct that functions with multiple water quality qPCR assays allowing for estimation of total fecal pollution and identification of key fecal sources. This study investigates SRM 2917 interlaboratory performance based on repeated measures of 12 qPCR assays by 14 laboratories (n = 1008 instrument runs). Using a Bayesian approach, single-instrument run data are combined to generate assay-specific global calibration models allowing for characterization of within- and between-lab variability. Comparable data sets generated by two additional laboratories are used to assess new SRM 2917 data acceptance metrics. SRM 2917 allows for reproducible single-instrument run calibration models across laboratories, regardless of qPCR assay. In addition, global models offer multiple data acceptance metric options that future users can employ to minimize variability, improve comparability of data across laboratories, and increase confidence in qPCR measurements.

Performance of NIST SRM® 2917 with 13 recreational water quality monitoring qPCR assays.

Fecal pollution remains a significant challenge for recreational water quality management worldwide. In response, there is a growing interest in the use of real-time quantitative PCR (qPCR) methods to achieve same-day notification of recreational water quality and associated public health risk as well as to characterize fecal pollution sources for targeted mitigation. However, successful widespread implementation of these technologies requires the development of and access to a high-quality standard control material. Here, we report a single laboratory qPCR performance assessment of the National Institute of Standards and Technology Standard Reference Material 2917 (NIST SRM® 2917), a linearized plasmid DNA construct that functions with 13 recreational water quality qPCR assays. Performance experiments indicate the generation of standard curves with amplification efficiencies ranging from 0.95 ± 0.006 to 0.99 ± 0.008 and coefficient of determination values (R2) ≥ 0.980. Regardless of qPCR assay, variability in repeated measurements at each dilution level were very low (quantification threshold standard deviations ≤ 0.657) and exhibited a heteroscedastic trend characteristic of qPCR standard curves. The influence of a yeast carrier tRNA added to the standard control material buffer was also investigated. Findings demonstrated that NIST SRM® 2917 functions with all qPCR methods and suggests that the future use of this control material by scientists and water quality managers should help reduce variability in concentration estimates and make results more consistent between laboratories.

Current progress and future opportunities in applications of bioinformatics for biodefense and pathogen detection: report from the Winter Mid-Atlantic Microbiome Meet-up, College Park, MD, January 10, 2018.

The Mid-Atlantic Microbiome Meet-up (M3) organization brings together academic, government, and industry groups to share ideas and develop best practices for microbiome research. In January of 2018, M3 held its fourth meeting, which focused on recent advances in biodefense, specifically those relating to infectious disease, and the use of metagenomic methods for pathogen detection. Presentations highlighted the utility of next-generation sequencing technologies for identifying and tracking microbial community members across space and time. However, they also stressed the current limitations of genomic approaches for biodefense, including insufficient sensitivity to detect low-abundance pathogens and the inability to quantify viable organisms. Participants discussed ways in which the community can improve software usability and shared new computational tools for metagenomic processing, assembly, annotation, and visualization. Looking to the future, they identified the need for better bioinformatics toolkits for longitudinal analyses, improved sample processing approaches for characterizing viruses and fungi, and more consistent maintenance of database resources. Finally, they addressed the necessity of improving data standards to incentivize data sharing. Here, we summarize the presentations and discussions from the meeting, identifying the areas where microbiome analyses have improved our ability to detect and manage biological threats and infectious disease, as well as gaps of knowledge in the field that require future funding and focus.

Rapid Prototyping of Nanofluidic Slits in a Silicone Bilayer.

This article reports a process for rapidly prototyping nanofluidic devices, particularly those comprising slits with microscale widths and nanoscale depths, in silicone. This process consists of designing a nanofluidic device, fabricating a photomask, fabricating a device mold in epoxy photoresist, molding a device in silicone, cutting and punching a molded silicone device, bonding a silicone device to a glass substrate, and filling the device with aqueous solution. By using a bilayer of hard and soft silicone, we have formed and filled nanofluidic slits with depths of less than 400 nm and aspect ratios of width to depth exceeding 250 without collapse of the slits. An important attribute of this article is that the description of this rapid prototyping process is very comprehensive, presenting context and details which are highly relevant to the rational implementation and reliable repetition of the process. Moreover, this process makes use of equipment commonly found in nanofabrication facilities and research laboratories, facilitating the broad adaptation and application of the process. Therefore, while this article specifically informs users of the Center for Nanoscale Science and Technology (CNST) at the National Institute of Standards and Technology (NIST), we anticipate that this information will be generally useful for the nanofabrication and nanofluidics research communities at large, and particularly useful for neophyte nanofabricators and nanofluidicists.

Total protein quantitation using the bicinchoninic acid assay and gradient elution moving boundary electrophoresis.

We investigated the ability of gradient elution moving boundary electrophoresis (GEMBE) with capacitively coupled contactless conductivity detection (C(4) D) to assay total protein concentration using the bicinchoninic acid (BCA) reaction. We chose this format because GEMBE-C(4) D behaves as a concentration dependent detection system, unlike optical methods that also rely on pathlength (due to Beer's law). This system tolerates proteins well compared with other capillary electrophoretic methods, allowing the capillary to be reused without coatings or additional hydroxide wash steps. The typical reaction protocol was modified by reducing the pH slightly from 11.25 to 9.4, which enabled elimination of tartrate from the reagents. We estimated that copper (I) could be detected at approximately 3.0 µmol/L, which agrees with similar GEMBE and CZE systems utilizing C(4) D. Under conditions similar to the BCA "micro method" assay, we determined the LOD for three common proteins (insulin, BSA, and bovine gamma globulin) and found that they agree well with the existing spectroscopic detection methods. Further, we investigated how long reaction times impact the LOD and found that the conversion was proportional to log(time). This indicated that little sensitivity is gained by extending the reaction past 1 h. Hence, GEMBE provides an alternative platform for total protein assays while maintaining the excellent sensitivity of the optical-based methods.

Capturing rare cells from blood using a packed bed of custom-synthesized chitosan microparticles.

We describe batch generation of uniform multifunctional chitosan microparticles for isolation of rare cells, such as circulating tumor cells (CTCs), from a sample of whole blood. The chitosan microparticles were produced in large numbers using a simple and inexpensive microtubing arrangement. The particles were functionalized through encapsulation of carbon black, to control autofluorescence, and surface attachment of streptavidin, to enable interactions with biotinylated antibodies. These large custom modified microparticles (≈164 µm diameter) were then packed into a microfluidic channel to demonstrate their utility in rare cell capture. Blood spiked with breast cancer (MCF-7) cells was first treated with a biotinylated antibody (anti-EpCAM, which is selective for cancer cells like MCF-7) and then pumped through the device. In the process, the cancer cells were selectively bound to the microparticles through non-covalent streptavidin-biotin interactions. The number density of captured cells was determined by fluorescence microscopy at physiologically relevant levels. Selective capture of the MCF-7 cells was characterized, and compared favorably with previous approaches. The overall approach using custom synthesized microparticles is versatile, and can allow researchers more flexibility for rare cell capture through simpler and cheaper methods than are currently employed.

Characterization of in vitro transcription amplification linearity and variability in the low copy number regime using External RNA Control Consortium (ERCC) spike-ins.

Using spike-in controls designed to mimic mammalian mRNA species, we used the quantitative reverse transcription polymerase chain reaction (RT-qPCR) to assess the performance of in vitro transcription (IVT) amplification process of small samples. We focused especially on the confidence of the transcript level measurement, which is essential for differential gene expression analyses. IVT reproduced gene expression profiles down to approximately 100 absolute input copies. However, a RT-qPCR analysis of the antisense RNA showed a systematic bias against low copy number transcripts, regardless of sequence. Experiments also showed that noise increases with decreasing copy number. First-round IVT preserved the gene expression information within a sample down to the 100 copy level, regardless of total input sample amount. However, the amplification was nonlinear under low total RNA input/long IVT conditions. Variability of the amplification increased predictably with decreasing input copy number. For the small enrichments of interest in typical differential gene expression studies (e.g., twofold changes), the bias from IVT reactions is unlikely to affect the results. In limited cases, some transcript-specific differential gene expression values will need adjustment to reflect this bias. Proper experimental design with reasonable detection limits will yield differential gene expression capability even between low copy number transcripts.

A simple packed bed device for antibody labelled rare cell capture from whole blood.

We have developed a system to isolate rare cells from whole blood using commercially available components and simple microfluidics. We characterized the capture of MCF-7 cells spiked into whole human blood using this system to demonstrate that enrichment and enumeration studies give results similar to in situ surface-modified devices while reducing fabrication and operation complexity.

Engineering and analysis of surface interactions in a microfluidic herringbone micromixer.

We developed a computational model and theoretical framework to investigate the geometrical optimization of particle-surface interactions in a herringbone micromixer. The enhancement of biomolecule- and particle-surface interactions in microfluidic devices through mixing and streamline disruption holds promise for a variety of applications. This analysis provides guidelines for optimizing the frequency and specific location of surface interactions based on the flow pattern and relative hydraulic resistance between a groove and the effective channel. The channel bottom, i.e., channel surface between grooves, was identified as the dominant location for surface contact. In addition, geometries that decrease the groove-to-channel hydraulic resistance improve contact with the channel top. Thus, herringbone mixers appear useful for a variety of surface-interaction applications, yet they have largely not been employed in an optimized fashion.

2D separations on a 1D chip: gradient elution moving boundary electrophoresis-chiral capillary zone electrophoresis.

A new method is described for two-dimensional (2D) separations using a microfluidic chip normally employed for single dimension electrophoresis. The method employs a combination of gradient elution moving boundary electrophoresis (GEMBE) and chiral capillary zone electrophoresis (CZE). The simplicity of the first dimension GEMBE method enables its implementation in the injection channel of a conventional electrophoresis chip, simplifying the design and operation of the device. The method was used for high resolution 2D chiral separations of a mixture of amino acids considered as possible signatures of extant or extinct life for solar system exploration. The enantiomers of aspartic acid, glutamic acid, serine, alanine, and valine were all resolved as well as glycine (achiral) and several unidentified impurities, giving an estimated peak capacity of 35 for the region between valine and glycine. The results highlight the need for high peak capacity separations for chiral amino acid analysis if accurate enantiomeric ratios are to be determined.

Image-based feedback control for real-time sorting of microspheres in a microfluidic device.

We describe a control system to automatically distribute antibody-functionalized beads to addressable assay chambers within a PDMS microfluidic device. The system used real-time image acquisition and processing to manage the valve states required to sort beads with unit precision. The image processing component of the control system correctly counted the number of beads in 99.81% of images (2689 of 2694), with only four instances of an incorrect number of beads being sorted to an assay chamber, and one instance of inaccurately counted beads being improperly delivered to waste. Post-experimental refinement of the counting script resulted in one counting error in 2694 images of beads (99.96% accuracy). We analyzed a range of operational variables (flow pressure, bead concentration, etc.) using a statistical model to characterize those that yielded optimal sorting speed and efficiency. The integrated device was able to capture, count, and deliver beads at a rate of approximately four per minute so that bead arrays could be assembled in 32 individually addressable assay chambers for eight analytical measurements in duplicate (512 beads total) within 2.5 hours. This functionality demonstrates the successful integration of a robust control system with precision bead handling that is the enabling technology for future development of a highly multiplexed bead-based analytical device.

T7-based linear amplification of low concentration mRNA samples using beads and microfluidics for global gene expression measurements.

We have demonstrated in vitro transcription (IVT) of cDNA sequences from purified Jurkat T-cell mRNA immobilized on microfluidic packed beds down to single-cell quantities. The microfluidically amplified antisense-RNA (aRNA) was nearly identical in length and quantity compared with benchtop reactions using the same starting sample quantities. Microarrays were used to characterize the number and population of genes in each sample, allowing comparison of the microfluidic and benchtop processes. For both benchtop and microfluidic assays, we measured the expression of approximately 4000 to 9000 genes for sample amounts ranging from 20 pg to 10 ng (2 to 1000 cell equivalents), corresponding to 41 to 93% of the absolute number of genes detected from a 100 ng total RNA control sample. Concordance of genes detected between methods (benchtop vs. microfluidic) and repeats (microfluidic vs. microfluidic) typically exceeded 90%. Validation of microarray by Real-time PCR of a panel of five genes suggests transcription of genes present is approximately six times more efficient with the microfluidic IVT compared with benchtop processing. Microfluidic IVT introduces no bias to the gene expression profile of the sample and provides more efficient transcription of mRNA sequences present at the single-cell level.

Simple device for multiplexed electrophoretic separations using gradient elution moving boundary electrophoresis with channel current detection.

A new microfluidic electrophoresis device and technique is described that is designed specifically for multiplexed, high-throughput separations. The device consists of an array of short (3 mm) capillaries connecting individual sample reservoirs to a common buffer reservoir. Each capillary in the array functions as both a separation channel and as a conductivity-based detection cell. The new technique is based upon the recently described gradient elution moving boundary electrophoresis (GEMBE) technique, which uses a combination of an electric field and buffer counterflow to achieve electrophoretic separations in short capillaries or microfluidic channels. A high voltage drives electrophoresis of the sample analytes through each separation channel. At the start of a separation, the bulk counterflow of buffer through the channel is high, and none of the analytes of interest can enter the channel. The counterflow is then gradually reduced until each analyte, in turn, is able to enter the channel where it is detected as a moving boundary or step. With very short capillaries, only one step at a time is present in each capillary, and the electric current through the channels can then be used as the detector signal, without any extra detector hardware. The current vs time signal for each channel is then smoothed and differentiated to produce a set of simultaneous electropherograms. Because there is no light source or other added hardware required for detection, the system is simple and can be easily and inexpensively scaled up to perform large numbers of simultaneous analyses. As a first demonstration, a 16-channel array device is used for high-throughput, time-series measurements of enzyme activity and inhibition.

Integrated continuous microfluidic liquid-liquid extraction.

We describe continuous flow liquid-liquid phase separation in microfluidic devices based on capillary forces and selective wetting surfaces. Effective liquid-liquid phase separation is achieved by using a thin porous fluoropolymer membrane that selectively wets non-aqueous solvents, has average pore sizes in the 0.1-1 microm range, and has a high pore density for high separation throughput. Pressure drops throughout the microfluidic network are modelled and operating regimes for the membrane phase separator are determined based on hydrodynamic pressure drops and capillary forces. A microfluidic extraction device integrating mixing and phase separation is realized by using silicon micromachining. Modeling of the phase separator establishes the operating limits. The device is capable of completely separating several organic-aqueous and fluorous-aqueous liquid-liquid systems, even with high fractions of partially miscible compounds. In each case, extraction is equivalent to one equilibrium extraction stage.

Continuous dielectrophoretic size-based particle sorting.

Continuous-flow dielectrophoretic (DEP) particle separation based on size is demonstrated in a microfluidic device. Polystyrene microspheres suspended in a neutrally buoyant aqueous solution are used as model particles to study DEP induced by an array of slanted, planar, interdigitated electrodes inside of a soft-lithography microchannel. The E-field gradients from the slanted electrodes impart a net transverse force component on the particles that causes them to "ratchet" across the channel. Over the length of the device, larger particles are deflected more than smaller particles according to the balance of hydrodynamic drag and DEP forces. Consequently, a flow-focused particle suspension containing different-sized particles is fractionated as the beads flow and separate down the length of the device. The flow behavior of spherical particles is modeled, and the total transverse particle displacement in the microfluidic device predicts fourth-order size and voltage and second-order inverse flow rate dependences. The model is verified experimentally for a range of flow rates, particle sizes, and E-field strengths.

Surfactant-enhanced liquid-liquid extraction in microfluidic channels with inline electric-field enhanced coalescence.

Continuous microfluidic liquid-liquid extraction is realized in a microfluidic device by generating emulsions with large interfacial areas for mass transfer, and subsequently breaking these emulsions using electric fields into easily separated segments of immiscible liquids (plugs). The microfluidic device employs insulated electrodes in a potassium hydroxide-etched channel to create large electric fields (100 kV m(-1)) that drive coalescence of the emulsion phase. The result is a transition from disperse to slug flow that can then readily be separated by gravity. Extractions of phenol and p-nitrophenol from an aqueous to hexane-surfactant solution serve as model systems. In addition to the increased surface area in the emulsion, extraction efficiency is enhanced by reverse micelles resulting from the presence of surfactants. The surfactant concentration is varied approximately 1-10 wt% and a general two-parameter model is developed to quantify the extraction behavior and demonstrate the effectiveness of reverse micelle enhanced extraction.