Multiplex technologies for the assessment of minimal residual disease and low-level mutation detection in leukaemia: mass spectrometry versus next-generation sequencing.

With the focus of leukaemia management shifting to the implications of low-level disease burden, increasing attention is being paid on the development of highly sensitive methodologies required for detection. There are various techniques capable of identification of measurable residual disease (MRD) either evidencing as relevant mutation detection [e.g. nucleophosmin 1 (NPM1) mutation] or trace levels of leukaemic clonal populations. The vast majority of these methods only permit detection of a single clone or mutation. However, mass spectrometry and next-generation sequencing enable the interrogation of multiple genes simultaneously, facilitating a more complete genomic profile. In the present review, we explore the methodologies of both techniques in conjunction with the important advantages and limitations associated with each assay. We also highlight the evidence and the various instances where either technique has been used and propose future strategies for MRD detection.

Performance assessment of DNA sequencing platforms in the ABRF Next-Generation Sequencing Study.

Assessing the reproducibility, accuracy and utility of massively parallel DNA sequencing platforms remains an ongoing challenge. Here the Association of Biomolecular Resource Facilities (ABRF) Next-Generation Sequencing Study benchmarks the performance of a set of sequencing instruments (HiSeq/NovaSeq/paired-end 2 × 250-bp chemistry, Ion S5/Proton, PacBio circular consensus sequencing (CCS), Oxford Nanopore Technologies PromethION/MinION, BGISEQ-500/MGISEQ-2000 and GS111) on human and bacterial reference DNA samples. Among short-read instruments, HiSeq 4000 and X10 provided the most consistent, highest genome coverage, while BGI/MGISEQ provided the lowest sequencing error rates. The long-read instrument PacBio CCS had the highest reference-based mapping rate and lowest non-mapping rate. The two long-read platforms PacBio CCS and PromethION/MinION showed the best sequence mapping in repeat-rich areas and across homopolymers. NovaSeq 6000 using 2 × 250-bp read chemistry was the most robust instrument for capturing known insertion/deletion events. This study serves as a benchmark for current genomics technologies, as well as a resource to inform experimental design and next-generation sequencing variant calling.

Quality Assessment and Validation of High-Throughput Sequencing for Grapevine Virus Diagnostics.

Development of High-Throughput Sequencing (HTS), also known as next generation sequencing, revolutionized diagnostic research of plant viruses. HTS outperforms bioassays and molecular diagnostic assays that are used to screen domestic and quarantine grapevine materials in data throughput, cost, scalability, and detection of novel and highly variant virus species. However, before HTS-based assays can be routinely used for plant virus diagnostics, performance specifications need to be developed and assessed. In this study, we selected 18 virus-infected grapevines as a test panel for measuring performance characteristics of an HTS-based diagnostic assay. Total nucleic acid (TNA) was extracted from petioles and dormant canes of individual samples and constructed libraries were run on Illumina NextSeq 500 instrument using a 75-bp single-end read platform. Sensitivity was 98% measured over 264 distinct virus and viroid infections with a false discovery rate (FDR) of approximately 1 in 5 positives. The results also showed that combining a spring petiole test with a fall cane test increased sensitivity to 100% for this TNA HTS assay. To evaluate extraction methodology, these results were compared to parallel dsRNA extractions. In addition, in a more detailed dilution study, the TNA HTS assay described here consistently performed well down to a dilution of 5%. In that range, sensitivity was 98% with a corresponding FDR of approximately 1 in 5. Repeatability and reproducibility were assessed at 99% and 93%, respectively. The protocol, criteria, and performance levels described here may help to standardize HTS for quality assurance and accreditation purposes in plant quarantine or certification programs.

National external quality assessment for next-generation sequencing-based diagnostics of primary immunodeficiencies.

Dutch genome diagnostic centers (GDC) use next-generation sequencing (NGS)-based diagnostic applications for the diagnosis of primary immunodeficiencies (PIDs). The interpretation of genetic variants in many PIDs is complicated because of the phenotypic and genetic heterogeneity. To analyze uniformity of variant filtering, interpretation, and reporting in NGS-based diagnostics for PID, an external quality assessment was performed. Four main Dutch GDCs participated in the quality assessment. Unannotated variant call format (VCF) files of two PID patient analyses per laboratory were distributed among the four GDCs, analyzed, and interpreted (eight analyses in total). Variants that would be reported to the clinician and/or advised for further investigation were compared between the centers. A survey measuring the experiences of clinical laboratory geneticists was part of the study. Analysis of samples with confirmed diagnoses showed that all centers reported at least the variants classified as likely pathogenic (LP) or pathogenic (P) variants in all samples, except for variants in two genes (PSTPIP1 and BTK). The absence of clinical information complicated correct classification of variants. In this external quality assessment, the final interpretation and conclusions of the genetic analyses were uniform among the four participating genetic centers. Clinical and immunological data provided by a medical specialist are required to be able to draw proper conclusions from genetic data.

Quality assessment of a clinical next-generation sequencing melanoma panel within the Italian Melanoma Intergroup (IMI).

BACKGROUND: Identification of somatic mutations in key oncogenes in melanoma is important to lead the effective and efficient use of personalized anticancer treatment. Conventional methods focus on few genes per run and, therefore, are unable to screen for multiple genes simultaneously. The use of Next-Generation Sequencing (NGS) technologies enables sequencing of multiple cancer-driving genes in a single assay, with reduced costs and DNA quantity needed and increased mutation detection sensitivity. METHODS: We designed a customized IMI somatic gene panel for targeted sequencing of actionable melanoma mutations; this panel was tested on three different NGS platforms using 11 metastatic melanoma tissue samples in blinded manner between two EMQN quality certificated laboratory. RESULTS: The detection limit of our assay was set-up to a Variant Allele Frequency (VAF) of 10% with a coverage of at least 200x. All somatic variants detected by all NGS platforms with a VAF ≥ 10%, were also validated by an independent method. The IMI panel achieved a very good concordance among the three NGS platforms. CONCLUSION: This study demonstrated that, using the main sequencing platforms currently available in the diagnostic setting, the IMI panel can be adopted among different centers providing comparable results.

Framework for quality assessment of whole genome cancer sequences.

Bringing together cancer genomes from different projects increases power and allows the investigation of pan-cancer, molecular mechanisms. However, working with whole genomes sequenced over several years in different sequencing centres requires a framework to compare the quality of these sequences. We used the Pan-Cancer Analysis of Whole Genomes cohort as a test case to construct such a framework. This cohort contains whole cancer genomes of 2832 donors from 18 sequencing centres. We developed a non-redundant set of five quality control (QC) measurements to establish a star rating system. These QC measures reflect known differences in sequencing protocol and provide a guide to downstream analyses and allow for exclusion of samples of poor quality. We have found that this is an effective framework of quality measures. The implementation of the framework is available at: https://dockstore.org/containers/quay.io/jwerner_dkfz/pancanqc:1.2.2 .

DeepVariant-on-Spark: Small-Scale Genome Analysis Using a Cloud-Based Computing Framework.

Although sequencing a human genome has become affordable, identifying genetic variants from whole-genome sequence data is still a hurdle for researchers without adequate computing equipment or bioinformatics support. GATK is a gold standard method for the identification of genetic variants and has been widely used in genome projects and population genetic studies for many years. This was until the Google Brain team developed a new method, DeepVariant, which utilizes deep neural networks to construct an image classification model to identify genetic variants. However, the superior accuracy of DeepVariant comes at the cost of computational intensity, largely constraining its applications. Accordingly, we present DeepVariant-on-Spark to optimize resource allocation, enable multi-GPU support, and accelerate the processing of the DeepVariant pipeline. To make DeepVariant-on-Spark more accessible to everyone, we have deployed the DeepVariant-on-Spark to the Google Cloud Platform (GCP). Users can deploy DeepVariant-on-Spark on the GCP following our instruction within 20 minutes and start to analyze at least ten whole-genome sequencing datasets using free credits provided by the GCP. DeepVaraint-on-Spark is freely available for small-scale genome analysis using a cloud-based computing framework, which is suitable for pilot testing or preliminary study, while reserving the flexibility and scalability for large-scale sequencing projects.

Development and Application of Performance Assessment Criteria for Next-Generation Sequencing-Based HIV Drug Resistance Assays.

Next-generation sequencing (NGS)-based HIV drug resistance (HIVDR) assays outperform conventional Sanger sequencing in scalability, sensitivity, and quantitative detection of minority resistance variants. Thus far, HIVDR assays have been applied primarily in research but rarely in clinical settings. One main obstacle is the lack of standardized validation and performance evaluation systems that allow regulatory agencies to benchmark and accredit new assays for clinical use. By revisiting the existing principles for molecular assay validation, here we propose a new validation and performance evaluation system that helps to both qualitatively and quantitatively assess the performance of an NGS-based HIVDR assay. To accomplish this, we constructed a 70-specimen proficiency test panel that includes plasmid mixtures at known ratios, viral RNA from infectious clones, and anonymized clinical specimens. We developed assessment criteria and benchmarks for NGS-based HIVDR assays and used these to assess data from five separate MiSeq runs performed in two experienced HIVDR laboratories. This proposed platform may help to pave the way for the standardization of NGS HIVDR assay validation and performance evaluation strategies for accreditation and quality assurance purposes in both research and clinical settings.

Ultralow-input single-tube linked-read library method enables short-read second-generation sequencing systems to routinely generate highly accurate and economical long-range sequencing information.

Long-range sequencing information is required for haplotype phasing, de novo assembly, and structural variation detection. Current long-read sequencing technologies can provide valuable long-range information but at a high cost with low accuracy and high DNA input requirements. We have developed a single-tube Transposase Enzyme Linked Long-read Sequencing (TELL-seq) technology, which enables a low-cost, high-accuracy, and high-throughput short-read second-generation sequencer to generate over 100 kb of long-range sequencing information with as little as 0.1 ng input material. In a PCR tube, millions of clonally barcoded beads are used to uniquely barcode long DNA molecules in an open bulk reaction without dilution and compartmentation. The barcoded linked-reads are used to successfully assemble genomes ranging from microbes to human. These linked-reads also generate megabase-long phased blocks and provide a cost-effective tool for detecting structural variants in a genome, which are important to identify compound heterozygosity in recessive Mendelian diseases and discover genetic drivers and diagnostic biomarkers in cancers.

External Quality Assessment for Next-Generation Sequencing-Based HIV Drug Resistance Testing: Unique Requirements and Challenges.

Over the past decade, there has been an increase in the adoption of next generation sequencing (NGS) technologies for HIV drug resistance (HIVDR) testing. NGS far outweighs conventional Sanger sequencing as it has much higher throughput, lower cost when samples are batched and, most importantly, significantly higher sensitivities for variants present at low frequencies, which may have significant clinical implications. Despite the advantages of NGS, Sanger sequencing remains the gold standard for HIVDR testing, largely due to the lack of standardization of NGS-based HIVDR testing. One important aspect of standardization includes external quality assessment (EQA) strategies and programs. Current EQA for Sanger-based HIVDR testing includes proficiency testing where samples are sent to labs and the performance of the lab conducting such assays is evaluated. The current methods for Sanger-based EQA may not apply to NGS-based tests because of the fundamental differences in their technologies and outputs. Sanger-based genotyping reports drug resistance mutations (DRMs) data as dichotomous, whereas NGS-based HIVDR genotyping also reports DRMs as numerical data (percent abundance). Here we present an overview of the need to develop EQA for NGS-based HIVDR testing and some unique challenges that may be encountered.

External Quality Assessment Program for Next-Generation Sequencing-Based HIV Drug Resistance Testing: Logistical Considerations.

Next-generation sequencing (NGS) is likely to become the new standard method for HIV drug resistance (HIVDR) genotyping. Despite the significant advances in the development of wet-lab protocols and bioinformatic data processing pipelines, one often-missing critical component of an NGS HIVDR assay for clinical use is external quality assessment (EQA). EQA is essential for ensuring assay consistency and laboratory competency in performing routine biomedical assays, and the rollout of NGS HIVDR tests in clinical practice will require an EQA. In September 2019, the 2nd International Symposium on NGS HIVDR was held in Winnipeg, Canada. It convened a multidisciplinary panel of experts, including research scientists, clinicians, bioinformaticians, laboratory biologists, biostatisticians, and EQA experts. A themed discussion was conducted on EQA strategies towards such assays during the symposium. This article describes the logistical challenges identified and summarizes the opinions and recommendations derived from these discussions, which may inform the development of an inaugural EQA program for NGS HIVDR in the near future.

Ten simple rules for providing effective bioinformatics research support.

Life scientists are increasingly turning to high-throughput sequencing technologies in their research programs, owing to the enormous potential of these methods. In a parallel manner, the number of core facilities that provide bioinformatics support are also increasing. Notably, the generation of complex large datasets has necessitated the development of bioinformatics support core facilities that aid laboratory scientists with cost-effective and efficient data management, analysis, and interpretation. In this article, we address the challenges-related to communication, good laboratory practice, and data handling-that may be encountered in core support facilities when providing bioinformatics support, drawing on our own experiences working as support bioinformaticians on multidisciplinary research projects. Most importantly, the article proposes a list of guidelines that outline how these challenges can be preemptively avoided and effectively managed to increase the value of outputs to the end user, covering the entire research project lifecycle, including experimental design, data analysis, and management (i.e., sharing and storage). In addition, we highlight the importance of clear and transparent communication, comprehensive preparation, appropriate handling of samples and data using monitoring systems, and the employment of appropriate tools and standard operating procedures to provide effective bioinformatics support.

Choice of library size normalization and statistical methods for differential gene expression analysis in balanced two-group comparisons for RNA-seq studies.

BACKGROUND: High-throughput RNA sequencing (RNA-seq) has evolved as an important analytical tool in molecular biology. Although the utility and importance of this technique have grown, uncertainties regarding the proper analysis of RNA-seq data remain. Of primary concern, there is no consensus regarding which normalization and statistical methods are the most appropriate for analyzing this data. The lack of standardized analytical methods leads to uncertainties in data interpretation and study reproducibility, especially with studies reporting high false discovery rates. In this study, we compared a recently developed normalization method, UQ-pgQ2, with three of the most frequently used alternatives including RLE (relative log estimate), TMM (Trimmed-mean M values) and UQ (upper quartile normalization) in the analysis of RNA-seq data. We evaluated the performance of these methods for gene-level differential expression analysis by considering the factors, including: 1) normalization combined with the choice of a Wald test from DESeq2 and an exact test/QL (Quasi-likelihood) F-Test from edgeR; 2) sample sizes in two balanced two-group comparisons; and 3) sequencing read depths. RESULTS: Using the MAQC RNA-seq datasets with small sample replicates, we found that UQ-pgQ2 normalization combined with an exact test can achieve better performance in term of power and specificity in differential gene expression analysis. However, using an intra-group analysis of false positives from real and simulated data, we found that a Wald test performs better than an exact test when the number of sample replicates is large and that a QL F-test performs the best given sample sizes of 5, 10 and 15 for any normalization. The RLE, TMM and UQ methods performed similarly given a desired sample size. CONCLUSION: We found the UQ-pgQ2 method combined with an exact test/QL F-test is the best choice in order to control false positives when the sample size is small. When the sample size is large, UQ-pgQ2 with a QL F-test is a better choice for the type I error control in an intra-group analysis. We observed read depths have a minimal impact for differential gene expression analysis based on the simulated data.

Assessment of significant procedures in multigene molecular detection for breast cancer in clinical laboratories: from variant detection to targeted therapy.

BACKGROUND: In recent years, numerous novel targeted drugs against breast cancer have been developed because of the rapid progress in multigene molecular testing based on next-generation sequencing (NGS). However, it is a great challenge for clinicians to update the drug information timely, therefore necessitating that clinical laboratories provide adequate and comprehensive targeted drugs information to clinicians as a reference. The premise of providing this information is the accuracy of variant detection. Our study aimed to assess the entire process of variant detection, interpretation, and targeted therapy. METHODS: Laboratories were instructed to use routine methods for variant detection. The results were evaluated based on a predefined score system, and differences in variant interpretation were analyzed. Targeted drug information provided by laboratories was also summarized, and its accuracy and sufficiency were assessed. RESULTS: Overall, 90.1% (82/91) of the laboratories produced accurate results. 78.9% (15/19) of the errors were false positives or false negatives. Incorrect and insufficient drug information was mainly provided due to failure in timely database updating, inconsistencies with the detected mutations or given clinical information, and negligence during phase I clinical trials. To prioritize providing targeted drug information, laboratories collected data were based on different factors, including variant clinical significance, allele frequency, and variant positions in the signal pathway. CONCLUSION: The variant detection capability was satisfactory, but the ability to provide accuracy and comprehensive targeted drug information should be urgently improved. Our study summarized a completed NGS-based multigene molecular detection pipeline, aiming to better inform precision treatment for breast cancer patients.

Assessment of a metabarcoding approach for the characterisation of vector-borne bacteria in canines from Bangkok, Thailand.

BACKGROUND: Globally, bacterial vector-borne disease (VBD) exerts a large toll on dogs in terms of morbidity and mortality but nowhere is this more pronounced than in the tropics. Tropical environments permit a burgeoning diversity and abundance of ectoparasites some of which can transmit an extensive range of infectious agents, including bacteria, amongst others. Although some of these vector-borne bacteria are responsible for both animal and human diseases in the tropics, there is a scarcity of epidemiological investigation into these pathogens' prevalence. The situation is further exacerbated by frequent canine co-infection, complicating symptomatology that regular diagnostic techniques may miss or be unable to fully characterise. Such limitations draw attention to the need to develop screening tools capable of detecting a wide range of pathogens from a host simultaneously. RESULTS: Here, we detail the employment of a next-generation sequencing (NGS) metabarcoding methodology to screen for the spectrum of bacterial VBD that are infecting semi-domesticated dogs across temple communities in Bangkok, Thailand. Our NGS detection protocol was able to find high levels of Ehrlichia canis, Mycoplasma haemocanis and Anaplasma platys infection rates as well as less common pathogens, such as "Candidatus Mycoplasma haematoparvum", Mycoplasma turicensis and Bartonella spp. We also compared our high-throughput approach to conventional endpoint PCR methods, demonstrating an improved detection ability for some bacterial infections, such as A. platys but a reduced ability to detect Rickettsia. CONCLUSIONS: Our methodology demonstrated great strength at detecting coinfections of vector-borne bacteria and rare pathogens that are seldom screened for in canines in the tropics, highlighting its advantages over traditional diagnostics to better characterise bacterial pathogens in environments where there is a dearth of research.

FQStat: a parallel architecture for very high-speed assessment of sequencing quality metrics.

BACKGROUND: High throughput DNA/RNA sequencing has revolutionized biological and clinical research. Sequencing is widely used, and generates very large amounts of data, mainly due to reduced cost and advanced technologies. Quickly assessing the quality of giga-to-tera base levels of sequencing data has become a routine but important task. Identification and elimination of low-quality sequence data is crucial for reliability of downstream analysis results. There is a need for a high-speed tool that uses optimized parallel programming for batch processing and simply gauges the quality of sequencing data from multiple datasets independent of any other processing steps. RESULTS: FQStat is a stand-alone, platform-independent software tool that assesses the quality of FASTQ files using parallel programming. Based on the machine architecture and input data, FQStat automatically determines the number of cores and the amount of memory to be allocated per file for optimum performance. Our results indicate that in a core-limited case, core assignment overhead exceeds the benefit of additional cores. In a core-unlimited case, there is a saturation point reached in performance by increasingly assigning additional cores per file. We also show that memory allocation per file has a lower priority in performance when compared to the allocation of cores. FQStat's output is summarized in HTML web page, tab-delimited text file, and high-resolution image formats. FQStat calculates and plots read count, read length, quality score, and high-quality base statistics. FQStat identifies and marks low-quality sequencing data to suggest removal from downstream analysis. We applied FQStat on real sequencing data to optimize performance and to demonstrate its capabilities. We also compared FQStat's performance to similar quality control (QC) tools that utilize parallel programming and attained improvements in run time. CONCLUSIONS: FQStat is a user-friendly tool with a graphical interface that employs a parallel programming architecture and automatically optimizes its performance to generate quality control statistics for sequencing data. Unlike existing tools, these statistics are calculated for multiple datasets and separately at the "lane," "sample," and "experiment" level to identify subsets of the samples with low quality, thereby preventing the loss of complete samples when reliable data can still be obtained.

Multilaboratory Assessment of a New Reference Material for Quality Assurance of Cell-Free Tumor DNA Measurements.

We conducted a multilaboratory assessment to determine the suitability of a new commercially available reference material with 40 cancer variants in a background of wild-type DNA at four different variant allele frequencies (VAFs): 2%, 0.50%, 0.125%, and 0%. The variants include single nucleotides, insertions, deletions, and two structural variations selected for their clinical importance and to challenge the performance of next-generation sequencing (NGS) methods. Fragmented DNA was formulated to simulate the size distribution of circulating wild-type and tumor DNA in a synthetic plasma matrix. DNA was extracted from these samples and characterized with different methods and multiple laboratories. The various extraction methods had differences in yield, perhaps because of differences in chemistry. Digital PCR assays were used to measure VAFs to compare results from different NGS methods. Comparable VAFs were observed across the different NGS methods. This multilaboratory assessment demonstrates that the new reference material is an appropriate tool to determine the analytical parameters of different measurement methods and to ensure their quality assurance.

In-depth comparative analysis of Illumina® MiSeq run metrics: Development of a wet-lab quality assessment tool.

Whole genome sequencing of bacterial isolates has become a daily task in many laboratories, generating incredible amounts of data. However, data acquisition is not an end in itself; the goal is to acquire high-quality data useful for understanding genetic relationships. Having a method that could rapidly determine which of the many available run metrics are the most important indicators of overall run quality and having a way to monitor these during a given sequencing run would be extremely helpful to this effect. Therefore, we compared various run metrics across 486 MiSeq runs, from five different machines. By performing a statistical analysis using principal components analysis and a K-means clustering algorithm of the metrics, we were able to validate metric comparisons among instruments, allowing for the development of a predictive algorithm, which permits one to observe whether a given MiSeq run has performed adequately. This algorithm is available in an Excel spreadsheet: that is, MiSeq Instrument & Run (In-Run) Forecast. Our tool can help verify that the quantity/quality of the generated sequencing data consistently meets or exceeds recommended manufacturer expectations. Patterns of deviation from those expectations can be used to assess potential run problems and plan preventative maintenance, which can save valuable time and funding resources.

Mitochondrial and nuclear disease panel (Mito-aND-Panel): Combined sequencing of mitochondrial and nuclear DNA by a cost-effective and sensitive NGS-based method.

BACKGROUND: The diagnosis of mitochondrial disorders is challenging because of the clinical variability and genetic heterogeneity of these conditions. Next-Generation Sequencing (NGS) technology offers a robust high-throughput platform for nuclear and mitochondrial DNA (mtDNA) analyses. METHOD: We developed a custom Agilent SureSelect Mitochondrial and Nuclear Disease Panel (Mito-aND-Panel) capture kit that allows parallel enrichment for subsequent NGS-based sequence analysis of nuclear mitochondrial disease-related genes and the complete mtDNA genome. Sequencing of enriched mtDNA simultaneously with nuclear genes was compared with the separated sequencing of the mitochondrial genome and whole exome sequencing (WES). RESULTS: The Mito-aND-Panel permits accurate detection of low-level mtDNA heteroplasmy due to a very high sequencing depth compared to standard diagnostic procedures using Sanger sequencing/SNaPshot and WES which is crucial to identify maternally inherited mitochondrial disorders. CONCLUSION: We established a NGS-based method with combined sequencing of the complete mtDNA and nuclear genes which enables a more sensitive heteroplasmy detection of mtDNA mutations compared to traditional methods. Because the method promotes the analysis of mtDNA variants in large cohorts, it is cost-effective and simple to setup, we anticipate this is a highly relevant method for sequence-based genetic diagnosis in clinical diagnostic applications.

Methodological Issues in Assessing the Economic Value of Next-Generation Sequencing Tests: Many Challenges and Not Enough Solutions.

BACKGROUND: Clinical use of next-generation sequencing (NGS) tests has been increasing, but few studies have examined their economic value. Several studies have noted that there are methodological challenges to conducting economic evaluations of NGS tests. OBJECTIVE: Our objective was to examine key methodological challenges for conducting economic evaluations of NGS tests, prioritize these challenges for future research, and identify how studies have attempted solutions to address these challenges. METHODS: We identified challenges for economic evaluations of NGS tests using prior literature and expert judgment of the co-authors. We used a modified Delphi assessment to prioritize challenges, based on importance and probability of resolution. Using a structured literature review and article extraction we then assessed whether published economic evaluations had addressed these challenges. RESULTS: We identified 11 challenges for conducting economic evaluations of NGS tests. The experts identified three challenges as the top priorities for future research: complex model structure, timeframe, and type of analysis and comparators used. Of the 15 published studies included in our literature review, four studies described specific solutions relevant to five of the 11 identified challenges. CONCLUSIONS: Major methodological challenges to economic evaluations of NGS tests remain to be addressed. Our results can be used to guide future research and inform decision-makers on how to prioritize research on the economic assessment of NGS tests.