Summarizing performance for genome scale measurement of miRNA: reference samples and metrics.

BACKGROUND: The potential utility of microRNA as biomarkers for early detection of cancer and other diseases is being investigated with genome-scale profiling of differentially expressed microRNA. Processes for measurement assurance are critical components of genome-scale measurements. Here, we evaluated the utility of a set of total RNA samples, designed with between-sample differences in the relative abundance of miRNAs, as process controls. RESULTS: Three pure total human RNA samples (brain, liver, and placenta) and two different mixtures of these components were evaluated as measurement assurance control samples on multiple measurement systems at multiple sites and over multiple rounds. In silico modeling of mixtures provided benchmark values for comparison with physical mixtures. Biomarker development laboratories using next-generation sequencing (NGS) or genome-scale hybridization assays participated in the study and returned data from the samples using their routine workflows. Multiplexed and single assay reverse-transcription PCR (RT-PCR) was used to confirm in silico predicted sample differences. Data visualizations and summary metrics for genome-scale miRNA profiling assessment were developed using this dataset, and a range of performance was observed. These metrics have been incorporated into an online data analysis pipeline and provide a convenient dashboard view of results from experiments following the described design. The website also serves as a repository for the accumulation of performance values providing new participants in the project an opportunity to learn what may be achievable with similar measurement processes. CONCLUSIONS: The set of reference samples used in this study provides benchmark values suitable for assessing genome-scale miRNA profiling processes. Incorporation of these metrics into an online resource allows laboratories to periodically evaluate their performance and assess any changes introduced into their measurement process.

In Vivo Site-Specific Protein Tagging with Diverse Amines Using an Engineered Sortase Variant.

Chemoenzymatic modification of proteins is an attractive option to create highly specific conjugates for therapeutics, diagnostics, or materials under gentle biological conditions. However, these methods often suffer from expensive specialized substrates, bulky fusion tags, low yields, and extra purification steps to achieve the desired conjugate. Staphylococcus aureus sortase A and its engineered variants are used to attach oligoglycine derivatives to the C-terminus of proteins expressed with a minimal LPXTG tag. This strategy has been used extensively for bioconjugation in vitro and for protein-protein conjugation in living cells. Here we show that an enzyme variant recently engineered for higher activity on oligoglycine has promiscuous activity that allows proteins to be tagged using a diverse array of small, commercially available amines, including several bioorthogonal functional groups. This technique can also be carried out in living Escherichia coli, enabling simple, inexpensive production of chemically functionalized proteins with no additional purification steps.

PEPR: pipelines for evaluating prokaryotic references.

The rapid adoption of microbial whole genome sequencing in public health, clinical testing, and forensic laboratories requires the use of validated measurement processes. Well-characterized, homogeneous, and stable microbial genomic reference materials can be used to evaluate measurement processes, improving confidence in microbial whole genome sequencing results. We have developed a reproducible and transparent bioinformatics tool, PEPR, Pipelines for Evaluating Prokaryotic References, for characterizing the reference genome of prokaryotic genomic materials. PEPR evaluates the quality, purity, and homogeneity of the reference material genome, and purity of the genomic material. The quality of the genome is evaluated using high coverage paired-end sequence data; coverage, paired-end read size and direction, as well as soft-clipping rates, are used to identify mis-assemblies. The homogeneity and purity of the material relative to the reference genome are characterized by comparing base calls from replicate datasets generated using multiple sequencing technologies. Genomic purity of the material is assessed by checking for DNA contaminants. We demonstrate the tool and its output using sequencing data while developing a Staphylococcus aureus candidate genomic reference material. PEPR is open source and available at https://github.com/usnistgov/pepr .

Characterization of in vitro transcription amplification linearity and variability in the low copy number regime using External RNA Control Consortium (ERCC) spike-ins.

Using spike-in controls designed to mimic mammalian mRNA species, we used the quantitative reverse transcription polymerase chain reaction (RT-qPCR) to assess the performance of in vitro transcription (IVT) amplification process of small samples. We focused especially on the confidence of the transcript level measurement, which is essential for differential gene expression analyses. IVT reproduced gene expression profiles down to approximately 100 absolute input copies. However, a RT-qPCR analysis of the antisense RNA showed a systematic bias against low copy number transcripts, regardless of sequence. Experiments also showed that noise increases with decreasing copy number. First-round IVT preserved the gene expression information within a sample down to the 100 copy level, regardless of total input sample amount. However, the amplification was nonlinear under low total RNA input/long IVT conditions. Variability of the amplification increased predictably with decreasing input copy number. For the small enrichments of interest in typical differential gene expression studies (e.g., twofold changes), the bias from IVT reactions is unlikely to affect the results. In limited cases, some transcript-specific differential gene expression values will need adjustment to reflect this bias. Proper experimental design with reasonable detection limits will yield differential gene expression capability even between low copy number transcripts.

Image-based feedback control for real-time sorting of microspheres in a microfluidic device.

We describe a control system to automatically distribute antibody-functionalized beads to addressable assay chambers within a PDMS microfluidic device. The system used real-time image acquisition and processing to manage the valve states required to sort beads with unit precision. The image processing component of the control system correctly counted the number of beads in 99.81% of images (2689 of 2694), with only four instances of an incorrect number of beads being sorted to an assay chamber, and one instance of inaccurately counted beads being improperly delivered to waste. Post-experimental refinement of the counting script resulted in one counting error in 2694 images of beads (99.96% accuracy). We analyzed a range of operational variables (flow pressure, bead concentration, etc.) using a statistical model to characterize those that yielded optimal sorting speed and efficiency. The integrated device was able to capture, count, and deliver beads at a rate of approximately four per minute so that bead arrays could be assembled in 32 individually addressable assay chambers for eight analytical measurements in duplicate (512 beads total) within 2.5 hours. This functionality demonstrates the successful integration of a robust control system with precision bead handling that is the enabling technology for future development of a highly multiplexed bead-based analytical device.

A framework for assessing 16S rRNA marker-gene survey data analysis methods using mixtures.

BACKGROUND: There are a variety of bioinformatic pipelines and downstream analysis methods for analyzing 16S rRNA marker-gene surveys. However, appropriate assessment datasets and metrics are needed as there is limited guidance to decide between available analysis methods. Mixtures of environmental samples are useful for assessing analysis methods as one can evaluate methods based on calculated expected values using unmixed sample measurements and the mixture design. Previous studies have used mixtures of environmental samples to assess other sequencing methods such as RNAseq. But no studies have used mixtures of environmental to assess 16S rRNA sequencing. RESULTS: We developed a framework for assessing 16S rRNA sequencing analysis methods which utilizes a novel two-sample titration mixture dataset and metrics to evaluate qualitative and quantitative characteristics of count tables. Our qualitative assessment evaluates feature presence/absence exploiting features only present in unmixed samples or titrations by testing if random sampling can account for their observed relative abundance. Our quantitative assessment evaluates feature relative and differential abundance by comparing observed and expected values. We demonstrated the framework by evaluating count tables generated with three commonly used bioinformatic pipelines: (i) DADA2 a sequence inference method, (ii) Mothur a de novo clustering method, and (iii) QIIME an open-reference clustering method. The qualitative assessment results indicated that the majority of Mothur and QIIME features only present in unmixed samples or titrations were accounted for by random sampling alone, but this was not the case for DADA2 features. Combined with count table sparsity (proportion of zero-valued cells in a count table), these results indicate DADA2 has a higher false-negative rate whereas Mothur and QIIME have higher false-positive rates. The quantitative assessment results indicated the observed relative abundance and differential abundance values were consistent with expected values for all three pipelines. CONCLUSIONS: We developed a novel framework for assessing 16S rRNA marker-gene survey methods and demonstrated the framework by evaluating count tables generated with three bioinformatic pipelines. This framework is a valuable community resource for assessing 16S rRNA marker-gene survey bioinformatic methods and will help scientists identify appropriate analysis methods for their marker-gene surveys.

Assembly and annotation of an Ashkenazi human reference genome.

BACKGROUND: Thousands of experiments and studies use the human reference genome as a resource each year. This single reference genome, GRCh38, is a mosaic created from a small number of individuals, representing a very small sample of the human population. There is a need for reference genomes from multiple human populations to avoid potential biases. RESULTS: Here, we describe the assembly and annotation of the genome of an Ashkenazi individual and the creation of a new, population-specific human reference genome. This genome is more contiguous and more complete than GRCh38, the latest version of the human reference genome, and is annotated with highly similar gene content. The Ashkenazi reference genome, Ash1, contains 2,973,118,650 nucleotides as compared to 2,937,639,212 in GRCh38. Annotation identified 20,157 protein-coding genes, of which 19,563 are > 99% identical to their counterparts on GRCh38. Most of the remaining genes have small differences. Forty of the protein-coding genes in GRCh38 are missing from Ash1; however, all of these genes are members of multi-gene families for which Ash1 contains other copies. Eleven genes appear on different chromosomes from their homologs in GRCh38. Alignment of DNA sequences from an unrelated Ashkenazi individual to Ash1 identified ~ 1 million fewer homozygous SNPs than alignment of those same sequences to the more-distant GRCh38 genome, illustrating one of the benefits of population-specific reference genomes. CONCLUSIONS: The Ash1 genome is presented as a reference for any genetic studies involving Ashkenazi Jewish individuals.

High-coverage, long-read sequencing of Han Chinese trio reference samples.

Single-molecule long-read sequencing datasets were generated for a son-father-mother trio of Han Chinese descent that is part of the Genome in a Bottle (GIAB) consortium portfolio. The dataset was generated using the Pacific Biosciences Sequel System. The son and each parent were sequenced to an average coverage of 60 and 30, respectively, with N50 subread lengths between 16 and 18 kb. Raw reads and reads aligned to both the GRCh37 and GRCh38 are available at the NCBI GIAB ftp site (ftp://ftp-trace.ncbi.nlm.nih.gov/giab/ftp/data/ChineseTrio/). The GRCh38 aligned read data are archived in NCBI SRA (SRX4739017, SRX4739121, and SRX4739122). This dataset is available for anyone to develop and evaluate long-read bioinformatics methods.

Simultaneous RNA purification and size selection using on-chip isotachophoresis with an ionic spacer.

We present an on-chip method for the extraction of RNA within a specific size range from low-abundance samples. We use isotachophoresis (ITP) with an ionic spacer and a sieving matrix to enable size-selection with a high yield of RNA in the target size range. The spacer zone separates two concentrated ITP peaks, the first containing unwanted single nucleotides and the second focusing RNA of the target size range (2-35 nt). Our ITP method excludes >90% of single nucleotides and >65% of longer RNAs (>35 nt). Compared to size selection using gel electrophoresis, ITP-based size-selection yields a 2.2-fold increase in the amount of extracted RNAs within the target size range. We also demonstrate compatibility of the ITP-based size-selection with downstream next generation sequencing. On-chip ITP-prepared samples reveal higher reproducibility of transcript-specific measurements compared to samples size-selected by gel electrophoresis. Our method offers an attractive alternative to conventional sample preparation for sequencing with shorter assay time, higher extraction efficiency and reproducibility. Potential applications of ITP-based size-selection include sequencing-based analyses of small RNAs from low-abundance samples such as rare cell types, samples from fluorescence activated cell sorting (FACS), or limited clinical samples.

Use of Standard Reference Material 2242 (Relative Intensity Correction Standard for Raman Spectroscopy) for microarray scanner qualification.

As a critical component of any microarray experiment, scanner performance has the potential to contribute variability and bias, the magnitude of which is usually not quantified. Using Standard Reference Material (SRM) 2,242, which is certified for Raman spectral correction, for monitoring the microarray fluorescence at the two most commonly used wavelengths, our team at the National Institute of Standards and Technology (NIST) has developed a method to establish scanner performance, qualifying signal measurement in microarray experiments. SRM 2,242 exhibits the necessary photostability at the excitation wavelengths of 635 nm and 532 nm, which allows scanner signal stability monitoring, although it is not certified for use in this capacity. In the current study, instrument response was tracked day to day, confirming that changes observed in experimental arrays scanned are not due to changes in the scanner response. Signal intensity and signal-to-noise ratio (S/N) were tracked over time on three different scanners, indicating the utility of the SRM for scanner qualification.

Microarray Scanner Performance Over a Five-Week Period as Measured With Cy5 and Cy3 Serial Dilution Slides.

To investigate scanner performance and guide development of an instrument qualification method, slides with replicates of successive dilutions of cyanine 5 (Cy5) and cyanine 3 (Cy3) dyes (referred to as dye slides) were scanned. The successive dilutions form a dose-response curve from which performance can be assessed. The effects of a variety of factors, including the number of scans and slide storage conditions, on scanner performance over a five-week period were investigated and tracked with time series charts of dye signal intensity, signal-to-noise (S/N), signal background, slope, and limit of detection (LOD). Scanner drift was tracked with a known stable reference material, Standard Reference Material (SRM) 2242. The greatest effect on the figures of merit was the dye, with the Cy5 dye showing signs of degradation after one week of scanning independent of all other factors while the Cy3 dye remained relatively stable. Use of the charts to track scanner performance over time holds promise for development of a method for microarray scanner performance qualification. Although not a prescription for performance qualification, this introductory study provides sufficient information regarding the use of dye slides to enable the user to institute a preliminary test method.

Unbiased Fitness Estimation of Pooled Barcode or Amplicon Sequencing Studies.

Standard practice for phenotyping complex cell pools is to measure the fold enrichment of genotype-specific amplicons after a period of competitive growth. Here, we show that fold-enrichment measures cannot be compared across genotype pools with different fitness distributions. We develop a method to calculate an unbiased estimate of relative fitness by tracking abundances over several time points and show how to optimize experimental protocols to minimize fitness measurement error.

Multiplexed precision genome editing with trackable genomic barcodes in yeast.

Our understanding of how genotype controls phenotype is limited by the scale at which we can precisely alter the genome and assess the phenotypic consequences of each perturbation. Here we describe a CRISPR-Cas9-based method for multiplexed accurate genome editing with short, trackable, integrated cellular barcodes (MAGESTIC) in Saccharomyces cerevisiae. MAGESTIC uses array-synthesized guide-donor oligos for plasmid-based high-throughput editing and features genomic barcode integration to prevent plasmid barcode loss and to enable robust phenotyping. We demonstrate that editing efficiency can be increased more than fivefold by recruiting donor DNA to the site of breaks using the LexA-Fkh1p fusion protein. We performed saturation editing of the essential gene SEC14 and identified amino acids critical for chemical inhibition of lipid signaling. We also constructed thousands of natural genetic variants, characterized guide mismatch tolerance at the genome scale, and ascertained that cryptic Pol III termination elements substantially reduce guide efficacy. MAGESTIC will be broadly useful to uncover the genetic basis of phenotypes in yeast.

Transcript-based redefinition of grouped oligonucleotide probe sets using AceView: high-resolution annotation for microarrays.

BACKGROUND: Extracting biological information from high-density Affymetrix arrays is a multi-step process that begins with the accurate annotation of microarray probes. Shortfalls in the original Affymetrix probe annotation have been described; however, few studies have provided rigorous solutions for routine data analysis. RESULTS: Using AceView, a comprehensive human transcript database, we have reannotated the probes by matching them to RNA transcripts instead of genes. Based on this transcript-level annotation, a new probe set definition was created in which every probe in a probe set maps to a common set of AceView gene transcripts. In addition, using artificial data sets we identified that a minimal probe set size of 4 is necessary for reliable statistical summarization. We further demonstrate that applying the new probe set definition can detect specific transcript variants contributing to differential expression and it also improves cross-platform concordance. CONCLUSION: We conclude that our transcript-level reannotation and redefinition of probe sets complement the original Affymetrix design. Redefinitions introduce probe sets whose sizes may not support reliable statistical summarization; therefore, we advocate using our transcript-level mapping redefinition in a secondary analysis step rather than as a replacement. Knowing which specific transcripts are differentially expressed is important to properly design probe/primer pairs for validation purposes. For convenience, we have created custom chip-description-files (CDFs) and annotation files for our new probe set definitions that are compatible with Bioconductor, Affymetrix Expression Console or third party software.

Toward achieving harmonization in a nano-cytotoxicity assay measurement through an interlaboratory comparison study.

Development of reliable cell-based nanotoxicology assays is important for evaluation of potentially hazardous engineered nanomaterials. Challenges to producing a reliable assay protocol include working with nanoparticle dispersions and living cell lines, and the potential for nano-related interference effects. Here we demonstrate the use of a 96-well plate design with several measurement controls and an interlaboratory comparison study involving five laboratories to characterize the robustness of a nanocytotoxicity MTS cell viability assay based on the A549 cell line. The consensus EC50 values were 22.1 mg/L (95% confidence intervals 16.9 mg/L to 27.2 mg/L) and 52.6 mg/L (44.1 mg/L to 62.6 mg/L) for positively charged polystyrene nanoparticles for the serum-free and serum conditions, respectively, and 49.7 µmol/L (47.5 µmol/L to 51.5 µmol/L) and 77.0 µmol/L (54.3 µmol/L to 99.4 µmol/L) for positive chemical control cadmium sulfate for the serum-free and serum conditions, respectively. Results from the measurement controls can be used to evaluate the sources of variability and their relative magnitudes within and between laboratories. This information revealed steps of the protocol that may need to be modified to improve the overall robustness and precision. The results suggest that protocol details such as cell line ID, media exchange, cell handling, and nanoparticle dispersion are critical to ensure protocol robustness and comparability of nanocytotoxicity assay results. The combination of system control measurements and interlaboratory comparison data yielded insights that would not have been available by either approach by itself.

Development and Characterization of Reference Materials for Genetic Testing: Focus on Public Partnerships.

Characterized reference materials (RMs) are needed for clinical laboratory test development and validation, quality control procedures, and proficiency testing to assure their quality. In this article, we review the development and characterization of RMs for clinical molecular genetic tests. We describe various types of RMs and how to access and utilize them, especially focusing on the Genetic Testing Reference Materials Coordination Program (Get-RM) and the Genome in a Bottle (GIAB) Consortium. This review also reinforces the need for collaborative efforts in the clinical genetic testing community to develop additional RMs.

Best practices for evaluating single nucleotide variant calling methods for microbial genomics.

Innovations in sequencing technologies have allowed biologists to make incredible advances in understanding biological systems. As experience grows, researchers increasingly recognize that analyzing the wealth of data provided by these new sequencing platforms requires careful attention to detail for robust results. Thus far, much of the scientific Communit's focus for use in bacterial genomics has been on evaluating genome assembly algorithms and rigorously validating assembly program performance. Missing, however, is a focus on critical evaluation of variant callers for these genomes. Variant calling is essential for comparative genomics as it yields insights into nucleotide-level organismal differences. Variant calling is a multistep process with a host of potential error sources that may lead to incorrect variant calls. Identifying and resolving these incorrect calls is critical for bacterial genomics to advance. The goal of this review is to provide guidance on validating algorithms and pipelines used in variant calling for bacterial genomics. First, we will provide an overview of the variant calling procedures and the potential sources of error associated with the methods. We will then identify appropriate datasets for use in evaluating algorithms and describe statistical methods for evaluating algorithm performance. As variant calling moves from basic research to the applied setting, standardized methods for performance evaluation and reporting are required; it is our hope that this review provides the groundwork for the development of these standards.

Minimum information for reporting next generation sequence genotyping (MIRING): Guidelines for reporting HLA and KIR genotyping via next generation sequencing.

The development of next-generation sequencing (NGS) technologies for HLA and KIR genotyping is rapidly advancing knowledge of genetic variation of these highly polymorphic loci. NGS genotyping is poised to replace older methods for clinical use, but standard methods for reporting and exchanging these new, high quality genotype data are needed. The Immunogenomic NGS Consortium, a broad collaboration of histocompatibility and immunogenetics clinicians, researchers, instrument manufacturers and software developers, has developed the Minimum Information for Reporting Immunogenomic NGS Genotyping (MIRING) reporting guidelines. MIRING is a checklist that specifies the content of NGS genotyping results as well as a set of messaging guidelines for reporting the results. A MIRING message includes five categories of structured information - message annotation, reference context, full genotype, consensus sequence and novel polymorphism - and references to three categories of accessory information - NGS platform documentation, read processing documentation and primary data. These eight categories of information ensure the long-term portability and broad application of this NGS data for all current histocompatibility and immunogenetics use cases. In addition, MIRING can be extended to allow the reporting of genotype data generated using pre-NGS technologies. Because genotyping results reported using MIRING are easily updated in accordance with reference and nomenclature databases, MIRING represents a bold departure from previous methods of reporting HLA and KIR genotyping results, which have provided static and less-portable data. More information about MIRING can be found online at miring.immunogenomics.org.

Ontology analysis of global gene expression differences of human bone marrow stromal cells cultured on 3D scaffolds or 2D films.

Differences in gene expression of human bone marrow stromal cells (hBMSCs) during culture in three-dimensional (3D) nanofiber scaffolds or on two-dimensional (2D) films were investigated via pathway analysis of microarray mRNA expression profiles. Previous work has shown that hBMSC culture in nanofiber scaffolds can induce osteogenic differentiation in the absence of osteogenic supplements (OS). Analysis using ontology databases revealed that nanofibers and OS regulated similar pathways and that both were enriched for TGF-ß and cell-adhesion/ECM-receptor pathways. The most notable difference between the two was that nanofibers had stronger enrichment for cell-adhesion/ECM-receptor pathways. Comparison of nanofibers scaffolds with flat films yielded stronger differences in gene expression than comparison of nanofibers made from different polymers, suggesting that substrate structure had stronger effects on cell function than substrate polymer composition. These results demonstrate that physical (nanofibers) and biochemical (OS) signals regulate similar ontological pathways, suggesting that these cues use similar molecular mechanisms to control hBMSC differentiation.

The determination of stem cell fate by 3D scaffold structures through the control of cell shape.

Stem cell response to a library of scaffolds with varied 3D structures was investigated. Microarray screening revealed that each type of scaffold structure induced a unique gene expression signature in primary human bone marrow stromal cells (hBMSCs). Hierarchical cluster analysis showed that treatments sorted by scaffold structure and not by polymer chemistry suggesting that scaffold structure was more influential than scaffold composition. Further, the effects of scaffold structure on hBMSC function were mediated by cell shape. Of all the scaffolds tested, only scaffolds with a nanofibrous morphology were able to drive the hBMSCs down an osteogenic lineage in the absence of osteogenic supplements. Nanofiber scaffolds forced the hBMSCs to assume an elongated, highly branched morphology. This same morphology was seen in osteogenic controls where hBMSCs were cultured on flat polymer films in the presence of osteogenic supplements (OS). In contrast, hBMSCs cultured on flat polymer films in the absence of OS assumed a more rounded and less-branched morphology. These results indicate that cells are more sensitive to scaffold structure than previously appreciated and suggest that scaffold efficacy can be optimized by tailoring the scaffold structure to force cells into morphologies that direct them to differentiate down the desired lineage.