Variant calling and benchmarking in an era of complete human genome sequences.

Genetic variant calling from DNA sequencing has enabled understanding of germline variation in hundreds of thousands of humans. Sequencing technologies and variant-calling methods have advanced rapidly, routinely providing reliable variant calls in most of the human genome. We describe how advances in long reads, deep learning, de novo assembly and pangenomes have expanded access to variant calls in increasingly challenging, repetitive genomic regions, including medically relevant regions, and how new benchmark sets and benchmarking methods illuminate their strengths and limitations. Finally, we explore the possible future of more complete characterization of human genome variation in light of the recent completion of a telomere-to-telomere human genome reference assembly and human pangenomes, and we consider the innovations needed to benchmark their newly accessible repetitive regions and complex variants.

Single-cell quantification of ribosome occupancy in early mouse development.

Translation regulation is critical for early mammalian embryonic development1. However, previous studies had been restricted to bulk measurements2, precluding precise determination of translation regulation including allele-specific analyses. Here, to address this challenge, we developed a novel microfluidic isotachophoresis (ITP) approach, named RIBOsome profiling via ITP (Ribo-ITP), and characterized translation in single oocytes and embryos during early mouse development. We identified differential translation efficiency as a key mechanism regulating genes involved in centrosome organization and N6-methyladenosine modification of RNAs. Our high-coverage measurements enabled, to our knowledge, the first analysis of allele-specific ribosome engagement in early development. These led to the discovery of stage-specific differential engagement of zygotic RNAs with ribosomes and reduced translation efficiency of transcripts exhibiting allele-biased expression. By integrating our measurements with proteomics data, we discovered that ribosome occupancy in germinal vesicle-stage oocytes is the predominant determinant of protein abundance in the zygote. The Ribo-ITP approach will enable numerous applications by providing high-coverage and high-resolution ribosome occupancy measurements from ultra-low input samples including single cells.

Testing at scale during the COVID-19 pandemic.

Assembly and publication of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) genome in January 2020 enabled the immediate development of tests to detect the new virus. This began the largest global testing programme in history, in which hundreds of millions of individuals have been tested to date. The unprecedented scale of testing has driven innovation in the strategies, technologies and concepts that govern testing in public health. This Review describes the changing role of testing during the COVID-19 pandemic, including the use of genomic surveillance to track SARS-CoV-2 transmission around the world, the use of contact tracing to contain disease outbreaks and testing for the presence of the virus circulating in the environment. Despite these efforts, widespread community transmission has become entrenched in many countries and has required the testing of populations to identify and isolate infected individuals, many of whom are asymptomatic. The diagnostic and epidemiological principles that underpin such population-scale testing are also considered, as are the high-throughput and point-of-care technologies that make testing feasible on a massive scale.

One in seven pathogenic variants can be challenging to detect by NGS: an analysis of 450,000 patients with implications for clinical sensitivity and genetic test implementation.

PURPOSE: To evaluate the impact of technically challenging variants on the implementation, validation, and diagnostic yield of commonly used clinical genetic tests. Such variants include large indels, small copy-number variants (CNVs), complex alterations, and variants in low-complexity or segmentally duplicated regions. METHODS: An interlaboratory pilot study used synthetic specimens to assess detection of challenging variant types by various next-generation sequencing (NGS)-based workflows. One well-performing workflow was further validated and used in clinician-ordered testing of more than 450,000 patients. RESULTS: In the interlaboratory study, only 2 of 13 challenging variants were detected by all 10 workflows, and just 3 workflows detected all 13. Limitations were also observed among 11 less-challenging indels. In clinical testing, 21.6% of patients carried one or more pathogenic variants, of which 13.8% (17,561) were classified as technically challenging. These variants were of diverse types, affecting 556 of 1,217 genes across hereditary cancer, cardiovascular, neurological, pediatric, reproductive carrier screening, and other indicated tests. CONCLUSION: The analytic and clinical sensitivity of NGS workflows can vary considerably, particularly for prevalent, technically challenging variants. This can have important implications for the design and validation of tests (by laboratories) and the selection of tests (by clinicians) for a wide range of clinical indications.

A crowdsourced set of curated structural variants for the human genome.

A high quality benchmark for small variants encompassing 88 to 90% of the reference genome has been developed for seven Genome in a Bottle (GIAB) reference samples. However a reliable benchmark for large indels and structural variants (SVs) is more challenging. In this study, we manually curated 1235 SVs, which can ultimately be used to evaluate SV callers or train machine learning models. We developed a crowdsourcing app-SVCurator-to help GIAB curators manually review large indels and SVs within the human genome, and report their genotype and size accuracy. SVCurator displays images from short, long, and linked read sequencing data from the GIAB Ashkenazi Jewish Trio son [NIST RM 8391/HG002]. We asked curators to assign labels describing SV type (deletion or insertion), size accuracy, and genotype for 1235 putative insertions and deletions sampled from different size bins between 20 and 892,149 bp. 'Expert' curators were 93% concordant with each other, and 37 of the 61 curators had at least 78% concordance with a set of 'expert' curators. The curators were least concordant for complex SVs and SVs that had inaccurate breakpoints or size predictions. After filtering events with low concordance among curators, we produced high confidence labels for 935 events. The SVCurator crowdsourced labels were 94.5% concordant with the heuristic-based draft benchmark SV callset from GIAB. We found that curators can successfully evaluate putative SVs when given evidence from multiple sequencing technologies.

The Environmental Microbiology Minimum Information (EMMI) Guidelines: qPCR and dPCR Quality and Reporting for Environmental Microbiology.

Real-time quantitative polymerase chain reaction (qPCR) and digital PCR (dPCR) methods have revolutionized environmental microbiology, yielding quantitative organism-specific data of nucleic acid targets in the environment. Such data are essential for characterizing interactions and processes of microbial communities, assessing microbial contaminants in the environment (water, air, fomites), and developing interventions (water treatment, surface disinfection, air purification) to curb infectious disease transmission. However, our review of recent qPCR and dPCR literature in our field of health-related environmental microbiology showed that many researchers are not reporting necessary and sufficient controls and methods, which would serve to strengthen their study results and conclusions. Here, we describe the application, utility, and interpretation of the suite of controls needed to make high quality qPCR and dPCR measurements of microorganisms in the environment. Our presentation is organized by the discrete steps and operations typical of this measurement process. We propose systematic terminology to minimize ambiguity and aid comparisons among studies. Example schemes for batching and combining controls for efficient work flow are demonstrated. We describe critical reporting elements for enhancing data credibility, and we provide an element checklist in the Supporting Information. Additionally, we present several key principles in metrology as context for laboratories to devise their own quality assurance and quality control reporting framework. Following the EMMI guidelines will improve comparability and reproducibility among qPCR and dPCR studies in environmental microbiology, better inform engineering and public health actions for preventing disease transmission through environmental pathways, and for the most pressing issues in the discipline, focus the weight of evidence in the direction toward solutions.

Genome-wide reconstruction of complex structural variants using read clouds.

In read cloud approaches, microfluidic partitioning of long genomic DNA fragments and barcoding of shorter fragments derived from these fragments retains long-range information in short sequencing reads. This combination of short reads with long-range information represents a powerful alternative to single-molecule long-read sequencing. We develop Genome-wide Reconstruction of Complex Structural Variants (GROC-SVs) for SV detection and assembly from read cloud data and apply this method to Illumina-sequenced 10x Genomics sarcoma and breast cancer data sets. Compared with short-fragment sequencing, GROC-SVs substantially improves the specificity of breakpoint detection at comparable sensitivity. This approach also performs sequence assembly across multiple breakpoints simultaneously, enabling the reconstruction of events exhibiting remarkable complexity. We show that chromothriptic rearrangements occurred before copy number amplifications, and that rates of single-nucleotide variants and SVs are not correlated. Our results support the use of read cloud approaches to advance the characterization of large and complex structural variation.

Measurements of translation initiation from all 64 codons in E. coli.

Our understanding of translation underpins our capacity to engineer living systems. The canonical start codon (AUG) and a few near-cognates (GUG, UUG) are considered as the 'start codons' for translation initiation in Escherichia coli. Translation is typically not thought to initiate from the 61 remaining codons. Here, we quantified translation initiation of green fluorescent protein and nanoluciferase in E. coli from all 64 triplet codons and across a range of DNA copy number. We detected initiation of protein synthesis above measurement background for 47 codons. Translation from non-canonical start codons ranged from 0.007 to 3% relative to translation from AUG. Translation from 17 non-AUG codons exceeded the highest reported rates of non-cognate codon recognition. Translation initiation from non-canonical start codons may contribute to the synthesis of peptides in both natural and synthetic biological systems.

Summarizing performance for genome scale measurement of miRNA: reference samples and metrics.

BACKGROUND: The potential utility of microRNA as biomarkers for early detection of cancer and other diseases is being investigated with genome-scale profiling of differentially expressed microRNA. Processes for measurement assurance are critical components of genome-scale measurements. Here, we evaluated the utility of a set of total RNA samples, designed with between-sample differences in the relative abundance of miRNAs, as process controls. RESULTS: Three pure total human RNA samples (brain, liver, and placenta) and two different mixtures of these components were evaluated as measurement assurance control samples on multiple measurement systems at multiple sites and over multiple rounds. In silico modeling of mixtures provided benchmark values for comparison with physical mixtures. Biomarker development laboratories using next-generation sequencing (NGS) or genome-scale hybridization assays participated in the study and returned data from the samples using their routine workflows. Multiplexed and single assay reverse-transcription PCR (RT-PCR) was used to confirm in silico predicted sample differences. Data visualizations and summary metrics for genome-scale miRNA profiling assessment were developed using this dataset, and a range of performance was observed. These metrics have been incorporated into an online data analysis pipeline and provide a convenient dashboard view of results from experiments following the described design. The website also serves as a repository for the accumulation of performance values providing new participants in the project an opportunity to learn what may be achievable with similar measurement processes. CONCLUSIONS: The set of reference samples used in this study provides benchmark values suitable for assessing genome-scale miRNA profiling processes. Incorporation of these metrics into an online resource allows laboratories to periodically evaluate their performance and assess any changes introduced into their measurement process.

Cell-based reference samples designed with specific differences in microRNA biomarkers.

BACKGROUND: We demonstrate the feasibility of creating a pair of reference samples to be used as surrogates for clinical samples measured in either a research or clinical laboratory setting. The reference sample paradigm presented and evaluated here is designed to assess the capability of a measurement process to detect true differences between two biological samples. Cell-based reference samples can be created with a biomarker signature pattern designed in silico. Clinical laboratories working in regulated applications are required to participate in proficiency testing programs; research laboratories doing discovery typically do not. These reference samples can be used in proficiency tests or as process controls that allow a laboratory to evaluate and optimize its measurement systems, monitor performance over time (process drift), assess changes in protocols, reagents, and/or personnel, maintain standard operating procedures, and most importantly, provide evidence for quality results. RESULTS: The biomarkers of interest in this study are microRNAs (miRNAs), small non-coding RNAs involved in the regulation of gene expression. Multiple lung cancer associated cell lines were determined by reverse transcription (RT)-PCR to have sufficiently different miRNA profiles to serve as components in mixture designs as reference samples. In silico models based on the component profiles were used to predict miRNA abundance ratios between two different cell line mixtures, providing target values for profiles obtained from in vitro mixtures. Two reference sample types were tested: total RNA mixed after extraction from cell lines, and intact cells mixed prior to RNA extraction. MicroRNA profiling of a pair of samples composed of extracted RNA derived from these cell lines successfully replicated the target values. Mixtures of intact cells from these lines also approximated the target values, demonstrating potential utility as mimics for clinical specimens. Both designs demonstrated their utility as reference samples for inter- or intra-laboratory testing. CONCLUSIONS: Cell-based reference samples can be created for performance assessment of a measurement process from biomolecule extraction through quantitation. Although this study focused on miRNA profiling with RT-PCR using cell lines associated with lung cancer, the paradigm demonstrated here should be extendable to genome-scale platforms and other biomolecular endpoints.

svclassify: a method to establish benchmark structural variant calls.

BACKGROUND: The human genome contains variants ranging in size from small single nucleotide polymorphisms (SNPs) to large structural variants (SVs). High-quality benchmark small variant calls for the pilot National Institute of Standards and Technology (NIST) Reference Material (NA12878) have been developed by the Genome in a Bottle Consortium, but no similar high-quality benchmark SV calls exist for this genome. Since SV callers output highly discordant results, we developed methods to combine multiple forms of evidence from multiple sequencing technologies to classify candidate SVs into likely true or false positives. Our method (svclassify) calculates annotations from one or more aligned bam files from many high-throughput sequencing technologies, and then builds a one-class model using these annotations to classify candidate SVs as likely true or false positives. RESULTS: We first used pedigree analysis to develop a set of high-confidence breakpoint-resolved large deletions. We then used svclassify to cluster and classify these deletions as well as a set of high-confidence deletions from the 1000 Genomes Project and a set of breakpoint-resolved complex insertions from Spiral Genetics. We find that likely SVs cluster separately from likely non-SVs based on our annotations, and that the SVs cluster into different types of deletions. We then developed a supervised one-class classification method that uses a training set of random non-SV regions to determine whether candidate SVs have abnormal annotations different from most of the genome. To test this classification method, we use our pedigree-based breakpoint-resolved SVs, SVs validated by the 1000 Genomes Project, and assembly-based breakpoint-resolved insertions, along with semi-automated visualization using svviz. CONCLUSIONS: We find that candidate SVs with high scores from multiple technologies have high concordance with PCR validation and an orthogonal consensus method MetaSV (99.7 % concordant), and candidate SVs with low scores are questionable. We distribute a set of 2676 high-confidence deletions and 68 high-confidence insertions with high svclassify scores from these call sets for benchmarking SV callers. We expect these methods to be particularly useful for establishing high-confidence SV calls for benchmark samples that have been characterized by multiple technologies.

In Vivo Site-Specific Protein Tagging with Diverse Amines Using an Engineered Sortase Variant.

Chemoenzymatic modification of proteins is an attractive option to create highly specific conjugates for therapeutics, diagnostics, or materials under gentle biological conditions. However, these methods often suffer from expensive specialized substrates, bulky fusion tags, low yields, and extra purification steps to achieve the desired conjugate. Staphylococcus aureus sortase A and its engineered variants are used to attach oligoglycine derivatives to the C-terminus of proteins expressed with a minimal LPXTG tag. This strategy has been used extensively for bioconjugation in vitro and for protein-protein conjugation in living cells. Here we show that an enzyme variant recently engineered for higher activity on oligoglycine has promiscuous activity that allows proteins to be tagged using a diverse array of small, commercially available amines, including several bioorthogonal functional groups. This technique can also be carried out in living Escherichia coli, enabling simple, inexpensive production of chemically functionalized proteins with no additional purification steps.

Evaluation of the External RNA Controls Consortium (ERCC) reference material using a modified Latin square design.

BACKGROUND: Highly multiplexed assays for quantitation of RNA transcripts are being used in many areas of biology and medicine. Using data generated by these transcriptomic assays requires measurement assurance with appropriate controls. Methods to prototype and evaluate multiple RNA controls were developed as part of the External RNA Controls Consortium (ERCC) assessment process. These approaches included a modified Latin square design to provide a broad dynamic range of relative abundance with known differences between four complex pools of ERCC RNA transcripts spiked into a human liver total RNA background. RESULTS: ERCC pools were analyzed on four different microarray platforms: Agilent 1- and 2-color, Illumina bead, and NIAID lab-made spotted microarrays; and two different second-generation sequencing platforms: the Life Technologies 5500xl and the Illumina HiSeq 2500. Individual ERCC controls were assessed for reproducible performance in signal response to concentration among the platforms. Most demonstrated linear behavior if they were not located near one of the extremes of the dynamic range. Performance issues with any individual ERCC transcript could be attributed to detection limitations, platform-specific target probe issues, or potential mixing errors. Collectively, these pools of spike-in RNA controls were evaluated for suitability as surrogates for endogenous transcripts to interrogate the performance of the RNA measurement process of each platform. The controls were useful for establishing the dynamic range of the assay, as well as delineating the useable region of that range where differential expression measurements, expressed as ratios, would be expected to be accurate. CONCLUSIONS: The modified Latin square design presented here uses a composite testing scheme for the evaluation of multiple performance characteristics: linear performance of individual controls, signal response within dynamic range pools of controls, and ratio detection between pairs of dynamic range pools. This compact design provides an economical sample format for the evaluation of multiple external RNA controls within a single experiment per platform. These results indicate that well-designed pools of RNA controls, spiked into samples, provide measurement assurance for endogenous gene expression studies.

svviz: a read viewer for validating structural variants.

UNLABELLED: Visualizing read alignments is the most effective way to validate candidate structural variants (SVs) with existing data. We present svviz, a sequencing read visualizer for SVs that sorts and displays only reads relevant to a candidate SV. svviz works by searching input bam(s) for potentially relevant reads, realigning them against the inferred sequence of the putative variant allele as well as the reference allele and identifying reads that match one allele better than the other. Separate views of the two alleles are then displayed in a scrollable web browser view, enabling a more intuitive visualization of each allele, compared with the single reference genome-based view common to most current read browsers. The browser view facilitates examining the evidence for or against a putative variant, estimating zygosity, visualizing affected genomic annotations and manual refinement of breakpoints. svviz supports data from most modern sequencing platforms. AVAILABILITY AND IMPLEMENTATION: svviz is implemented in python and freely available from http://svviz.github.io/.

PEPR: pipelines for evaluating prokaryotic references.

The rapid adoption of microbial whole genome sequencing in public health, clinical testing, and forensic laboratories requires the use of validated measurement processes. Well-characterized, homogeneous, and stable microbial genomic reference materials can be used to evaluate measurement processes, improving confidence in microbial whole genome sequencing results. We have developed a reproducible and transparent bioinformatics tool, PEPR, Pipelines for Evaluating Prokaryotic References, for characterizing the reference genome of prokaryotic genomic materials. PEPR evaluates the quality, purity, and homogeneity of the reference material genome, and purity of the genomic material. The quality of the genome is evaluated using high coverage paired-end sequence data; coverage, paired-end read size and direction, as well as soft-clipping rates, are used to identify mis-assemblies. The homogeneity and purity of the material relative to the reference genome are characterized by comparing base calls from replicate datasets generated using multiple sequencing technologies. Genomic purity of the material is assessed by checking for DNA contaminants. We demonstrate the tool and its output using sequencing data while developing a Staphylococcus aureus candidate genomic reference material. PEPR is open source and available at https://github.com/usnistgov/pepr .

Using mixtures of biological samples as process controls for RNA-sequencing experiments.

BACKGROUND: Genome-scale "-omics" measurements are challenging to benchmark due to the enormous variety of unique biological molecules involved. Mixtures of previously-characterized samples can be used to benchmark repeatability and reproducibility using component proportions as truth for the measurement. We describe and evaluate experiments characterizing the performance of RNA-sequencing (RNA-Seq) measurements, and discuss cases where mixtures can serve as effective process controls. RESULTS: We apply a linear model to total RNA mixture samples in RNA-seq experiments. This model provides a context for performance benchmarking. The parameters of the model fit to experimental results can be evaluated to assess bias and variability of the measurement of a mixture. A linear model describes the behavior of mixture expression measures and provides a context for performance benchmarking. Residuals from fitting the model to experimental data can be used as a metric for evaluating the effect that an individual step in an experimental process has on the linear response function and precision of the underlying measurement while identifying signals affected by interference from other sources. Effective benchmarking requires well-defined mixtures, which for RNA-Seq requires knowledge of the post-enrichment 'target RNA' content of the individual total RNA components. We demonstrate and evaluate an experimental method suitable for use in genome-scale process control and lay out a method utilizing spike-in controls to determine enriched RNA content of total RNA in samples. CONCLUSIONS: Genome-scale process controls can be derived from mixtures. These controls relate prior knowledge of individual components to a complex mixture, allowing assessment of measurement performance. The target RNA fraction accounts for differential selection of RNA out of variable total RNA samples. Spike-in controls can be utilized to measure this relationship between target RNA content and input total RNA. Our mixture analysis method also enables estimation of the proportions of an unknown mixture, even when component-specific markers are not previously known, whenever pure components are measured alongside the mixture.

Use of Cause-and-Effect Analysis to Design a High-Quality Nanocytotoxicology Assay.

An important consideration in developing standards and regulations that govern the production and use of commercial nanoscale materials is the development of robust and reliable measurements to monitor the potential adverse biological effects of such products. These measurements typically require cell-based and other biological assays that provide an assessment of the risks associated with the nanomaterial of interest. In this perspective, we describe the use of cause-and-effect (C&E) analysis to design robust, high quality cell-based assays to test nanoparticle-related cytotoxicity. C&E analysis of an assay system identifies the sources of variability that influence the test result. These sources can then be used to design control experiments that aid in establishing the validity of a test result. We demonstrate the application of C&E analysis to the commonly used 3-(4,5-dimethylthiazol-2-yl)-5-(3-carboxymethoxyphenyl)-2-(4-sulfophenyl)-2H-tetrazolium (MTS) cell-viability assay. This is the first time to our knowledge that C&E analysis has been used to characterize a cell-based toxicity assay. We propose the use of a 96-well plate layout which incorporates a range of control experiments to assess multiple factors such as nanomaterial interference, pipetting accuracy, cell seeding density, and instrument performance, and demonstrate the performance of the assay using the plate layout in a case study. While the plate layout was formulated specifically for the MTS assay, it is applicable to other cytotoxicity, ecotoxicity (i.e., bacteria toxicity), and nanotoxicity assays after assay-specific modifications.