The Labyrinth of Product Development and Regulatory Approvals in Liquid Biopsy Diagnostics.

The evolution of chemistries and instrument platforms for next-generation sequencing has led to sequencing of genomic variants in both tumor biopsies as well as in circulating tumor cells (CTCs) and cell-free DNA liquid biopsies. The transition of these analytical platforms into clinical ones has led to challenges in product development as well as regulatory strategies for the approval of diagnostic products with these platforms. Regulatory strategies for liquid biopsy diagnostics depend on a framework that has been developed over the past few years by the US Food and Drug Administration (FDA). This framework includes both guidances that cover enrichment biomarkers and companion diagnostics, as well as regulatory approval precedents, which can be used to design regulatory strategies for new liquid biopsy diagnostic products. However, the regulatory paths for these liquid biopsy diagnostics can also be tortuous, as is the example of CTC-platform liquid biopsies. The ultimate success of regulatory pathways of liquid biopsy diagnostics has been driven by the incremental value of FDA approval for Clinical Laboratory Improvement Amendment (CLIA)-developed tests and by the inherent complexity of these diagnostics, which are practical barriers for the widespread replication of these tests throughout CLIA laboratories. The framework for FDA approval of sequence information from these liquid biopsies has been focused on single-site approvals of diagnostics where sequencing information is considered at different diagnostic risk levels, ranging from novel or follow-on companion diagnostics to variant calls in genomic targets considered independently valuable for therapeutic decision making.

Translational biomarkers: from preclinical to clinical a report of 2009 AAPS/ACCP Biomarker Workshop.

There have been some successes in qualifying biomarkers and applying them to drug development and clinical treatment of various diseases. A recent success is illustrated by a collaborative effort among the US Food and Drug Administration, the European Medicines Agency, and the pharmaceutical industry to provide a set of seven preclinical kidney toxicity biomarkers for drug development. Other successes include, but are not limited to, clinical biomarkers for cancer treatment and clinical management of heart transplant patients. The value of fully qualified surrogate endpoints in facilitating successful drug development is undisputed, especially for diseases in which the traditional clinical outcome can only be assessed in large, multi-year trials. Emerging biomarkers, including chemical genomic or imaging biomarkers, and measurement of circulating tumor cells hold great promise for early diagnosis of disease and as prognostic tests for managing treatment of chronic diseases such as osteoarthritis, Alzheimer disease, cardiovascular disease, and cancer. To advance the success of treating and managing these diseases, efforts are needed to establish the temporal relationship between changes in inflammatory or imaging biomarkers with the progression of the chronic disease, and in the case of cancer, between the extent of circulating cancer cells and tumor progression or remission.

Translational medicine and the value of biomarker qualification.

The gap between development of exploratory biomarkers and their acceptance in drug development and regulatory review is a hurdle in the development of better therapies. The U.S. Food and Drug Administration has developed a regulatory process for biomarker qualification to accelerate the process by which new biomarkers are integrated in the development of therapies.

The MicroArray Quality Control (MAQC)-II study of common practices for the development and validation of microarray-based predictive models.

Gene expression data from microarrays are being applied to predict preclinical and clinical endpoints, but the reliability of these predictions has not been established. In the MAQC-II project, 36 independent teams analyzed six microarray data sets to generate predictive models for classifying a sample with respect to one of 13 endpoints indicative of lung or liver toxicity in rodents, or of breast cancer, multiple myeloma or neuroblastoma in humans. In total, >30,000 models were built using many combinations of analytical methods. The teams generated predictive models without knowing the biological meaning of some of the endpoints and, to mimic clinical reality, tested the models on data that had not been used for training. We found that model performance depended largely on the endpoint and team proficiency and that different approaches generated models of similar performance. The conclusions and recommendations from MAQC-II should be useful for regulatory agencies, study committees and independent investigators that evaluate methods for global gene expression analysis.

Voluntary exploratory data submissions to the US FDA and the EMA: experience and impact.

Heterogeneity in the underlying mechanisms of disease processes and inter-patient variability in drug responses are major challenges in drug development. To address these challenges, biomarker strategies based on a range of platforms, such as microarray gene-expression technologies, are increasingly being applied to elucidate these sources of variability and thereby potentially increase drug development success rates. With the aim of enhancing understanding of the regulatory significance of such biomarker data by regulators and sponsors, the US Food and Drug Administration initiated a programme in 2004 to allow sponsors to submit exploratory genomic data voluntarily, without immediate regulatory impact. In this article, a selection of case studies from the first 5 years of this programme - which is now known as the voluntary exploratory data submission programme, and also involves collaboration with the European Medicines Agency - are discussed, and general lessons are highlighted.

The balance of reproducibility, sensitivity, and specificity of lists of differentially expressed genes in microarray studies.

BACKGROUND: Reproducibility is a fundamental requirement in scientific experiments. Some recent publications have claimed that microarrays are unreliable because lists of differentially expressed genes (DEGs) are not reproducible in similar experiments. Meanwhile, new statistical methods for identifying DEGs continue to appear in the scientific literature. The resultant variety of existing and emerging methods exacerbates confusion and continuing debate in the microarray community on the appropriate choice of methods for identifying reliable DEG lists. RESULTS: Using the data sets generated by the MicroArray Quality Control (MAQC) project, we investigated the impact on the reproducibility of DEG lists of a few widely used gene selection procedures. We present comprehensive results from inter-site comparisons using the same microarray platform, cross-platform comparisons using multiple microarray platforms, and comparisons between microarray results and those from TaqMan - the widely regarded "standard" gene expression platform. Our results demonstrate that (1) previously reported discordance between DEG lists could simply result from ranking and selecting DEGs solely by statistical significance (P) derived from widely used simple t-tests; (2) when fold change (FC) is used as the ranking criterion with a non-stringent P-value cutoff filtering, the DEG lists become much more reproducible, especially when fewer genes are selected as differentially expressed, as is the case in most microarray studies; and (3) the instability of short DEG lists solely based on P-value ranking is an expected mathematical consequence of the high variability of the t-values; the more stringent the P-value threshold, the less reproducible the DEG list is. These observations are also consistent with results from extensive simulation calculations. CONCLUSION: We recommend the use of FC-ranking plus a non-stringent P cutoff as a straightforward and baseline practice in order to generate more reproducible DEG lists. Specifically, the P-value cutoff should not be stringent (too small) and FC should be as large as possible. Our results provide practical guidance to choose the appropriate FC and P-value cutoffs when selecting a given number of DEGs. The FC criterion enhances reproducibility, whereas the P criterion balances sensitivity and specificity.

Histopathology of vascular injury in Sprague-Dawley rats treated with phosphodiesterase IV inhibitor SCH 351591 or SCH 534385.

Histopathological and immunohistochemical studies were conducted to characterize vascular injuries in rats treated with phosphodiesterase (PDE) IV inhibitors SCH 351591 or SCH 534385. Sprague-Dawley rats were administered PDE IV inhibitors by gavage at a range of doses and times. The two PDE IV inhibitors induced comparable levels of vascular injury, primarily in the mesentery and to a lesser extent in the pancreas, kidney, liver, small intestine, and stomach. Mesenteric vascular changes occurred as early as one hour, progressively developed over twenty-four to forty-eight hours, peaked at seventy-two hours, and gradually subsided from seven to nine days. The typical morphology of the vascular toxicity consisted of hemorrhage and necrosis of arterioles and arteries, microvascular injury, fibrin deposition, and perivascular inflammation of a variety of blood vessels. The incidence and severity of mesenteric vascular injury increased in a time- and dose-dependent manner in SCH 351591- or SCH 534385-treated rats. Mesenteric vascular injury was frequently associated with activation of mast cells (MC), endothelial cells (EC), and macrophages (MØ). Immunohistochemical studies showed increases in CD63 immunoreactivity of mesenteric MC and in nitrotyrosine immunoreactivity of mesenteric EC and MØ. The present study also provides a morphological and cellular basis for evaluating candidate biomarkers of drug-induced vascular injury.

Biomarkers in peripheral blood associated with vascular injury in Sprague-Dawley rats treated with the phosphodiesterase IV inhibitors SCH 351591 or SCH 534385.

Drug-associated vascular injury can be caused by phosphodiesterase (PDE) IV inhibitors and drugs from several other classes. The pathogenesis is poorly understood, but it appears to include vascular and innate immunological components. This research was undertaken to identify changes in peripheral blood associated with vascular injury caused by PDE IV inhibitors. We evaluated twelve proteins, serum nitrite, and leukocyte populations in peripheral blood of rats treated with experimental PDE IV inhibitors. We found that these compounds produced histological microvascular injury in a dose- and time-dependent manner. Measurement of these serum proteins showed changes in eight of the twelve examined. Changes were seen in the levels of: tissue inhibitor of metalloproteinase-1, alpha1-acid glycoprotein, GRO/CINC-1, vascular endothelial growth factor, C-reactive protein, haptoglobin, thrombomodulin, and interleukin-6. No changes were seen in levels of tumor necrosis factor-alpha, hepatocyte growth factor, nerve growth factor, and granulocyte-monocyte colony stimulating factor. Serum levels of nitrite were also increased. Circulating granulocyte numbers were increased, and lymphocyte numbers were decreased. The changes in these parameters showed both a dose- and time-dependent association with histopathologic changes. These biomarkers could provide an additional tool for the nonclinical and clinical evaluation of investigational compounds.

Strategic paths for biomarker qualification.

Biomarkers may be qualified using different qualification processes. A passive approach for qualification has been to accept the end of discussions in the scientific literature as an indication that a biomarker has been accepted. An active approach to qualification requires development of a comprehensive process by which a consensus may be reached about the qualification of a biomarker. Active strategies for qualification include those associated with context-independent as well as context-dependent qualifications.

Qualification of biomarkers for drug development in organ transplantation.

The drug development process is dependent upon having established end points for measuring drug efficacy and adverse effects. New drug development in organ transplantation suffers from having end points which are either outdated or which do not serve the purpose of addressing the current critical drug therapy problems. Numerous biomarkers have been examined in organ transplantation, but almost all would be classified as exploratory for drug development purposes. Some of the possible pathways out of this dilemma include investigator- or consortium-initiated research that would qualify the biomarkers as either probable or known valid biomarkers, help in identification of new end points in transplantation and their associated biomarkers, co-development of a new biomarker and drug for transplantation and the use of new clinical trial design methods which facilitate enriched or stratified transplant patient populations. With new biomarkers and new study design methodologies for drug development, improvement in the drug development process for transplantation is a real possibility that the transplant clinical and research community can help to bring about.

Questions and answers about the Pilot Process for Biomarker Qualification at the FDA.

The FDA has developed a Pilot Process for Biomarker Qualification. Initial experience with this process has underscored the care that a long-term approach to biomarker qualification independently of development for specific drugs should have in the assembly of external industry consortia as well as the internal regulatory organization for these qualification efforts. There are complex scientific and clinical issues associated with these qualifications, and it is paramount that the expertise needed for their review be identified so that a comprehensive consensus may be reached at the end of this process. Several frequently asked questions associated with this process are presented in this paper, as well as answers reflecting the Agency's current thinking about this process.:

The Predictive Safety Testing Consortium: A synthesis of the goals, challenges and accomplishments of the Critical Path.

The qualification of biomarkers of drug safety requires data on many compounds and nonclinical and clinical studies. The cost and effort associated with these qualifications cannot be easily covered by a single pharmaceutical company. Intellectual property associated with safety biomarkers is also held by many different companies. Consortia between different pharmaceutical companies can overcome cost and intellectual property hurdles to biomarker qualification. The Predictive Safety Testing Consortium (PSTC) is a collaborative effort between 16 different pharmaceutical companies to generate data supporting biomarker qualification. This Consortium is coordinated through the C-Path Institute, and currently has five biomarker qualification working groups engaged in this collaboration: nephrotoxicity, hepatotoxicity, vascular injury, myopathy, and non-genotoxic carcinogenicity. These working groups are aided by a data management team and a translational strategy team. Qualification studies of promising biomarkers are already progressing in several of the working groups, and results in the nephrotoxicity working group warranted a data submission to the FDA and EMEA for regulatory qualification of new nephrotoxicity biomarkers.:

Rat toxicogenomic study reveals analytical consistency across microarray platforms.

To validate and extend the findings of the MicroArray Quality Control (MAQC) project, a biologically relevant toxicogenomics data set was generated using 36 RNA samples from rats treated with three chemicals (aristolochic acid, riddelliine and comfrey) and each sample was hybridized to four microarray platforms. The MAQC project assessed concordance in intersite and cross-platform comparisons and the impact of gene selection methods on the reproducibility of profiling data in terms of differentially expressed genes using distinct reference RNA samples. The real-world toxicogenomic data set reported here showed high concordance in intersite and cross-platform comparisons. Further, gene lists generated by fold-change ranking were more reproducible than those obtained by t-test P value or Significance Analysis of Microarrays. Finally, gene lists generated by fold-change ranking with a nonstringent P-value cutoff showed increased consistency in Gene Ontology terms and pathways, and hence the biological impact of chemical exposure could be reliably deduced from all platforms analyzed.

Evaluation of DNA microarray results with quantitative gene expression platforms.

We have evaluated the performance characteristics of three quantitative gene expression technologies and correlated their expression measurements to those of five commercial microarray platforms, based on the MicroArray Quality Control (MAQC) data set. The limit of detection, assay range, precision, accuracy and fold-change correlations were assessed for 997 TaqMan Gene Expression Assays, 205 Standardized RT (Sta)RT-PCR assays and 244 QuantiGene assays. TaqMan is a registered trademark of Roche Molecular Systems, Inc. We observed high correlation between quantitative gene expression values and microarray platform results and found few discordant measurements among all platforms. The main cause of variability was differences in probe sequence and thus target location. A second source of variability was the limited and variable sensitivity of the different microarray platforms for detecting weakly expressed genes, which affected interplatform and intersite reproducibility of differentially expressed genes. From this analysis, we conclude that the MAQC microarray data set has been validated by alternative quantitative gene expression platforms thus supporting the use of microarray platforms for the quantitative characterization of gene expression.

Microarray scanner calibration curves: characteristics and implications.

BACKGROUND: Microarray-based measurement of mRNA abundance assumes a linear relationship between the fluorescence intensity and the dye concentration. In reality, however, the calibration curve can be nonlinear. RESULTS: By scanning a microarray scanner calibration slide containing known concentrations of fluorescent dyes under 18 PMT gains, we were able to evaluate the differences in calibration characteristics of Cy5 and Cy3. First, the calibration curve for the same dye under the same PMT gain is nonlinear at both the high and low intensity ends. Second, the degree of nonlinearity of the calibration curve depends on the PMT gain. Third, the two PMTs (for Cy5 and Cy3) behave differently even under the same gain. Fourth, the background intensity for the Cy3 channel is higher than that for the Cy5 channel. The impact of such characteristics on the accuracy and reproducibility of measured mRNA abundance and the calculated ratios was demonstrated. Combined with simulation results, we provided explanations to the existence of ratio underestimation, intensity-dependence of ratio bias, and anti-correlation of ratios in dye-swap replicates. We further demonstrated that although Lowess normalization effectively eliminates the intensity-dependence of ratio bias, the systematic deviation from true ratios largely remained. A method of calculating ratios based on concentrations estimated from the calibration curves was proposed for correcting ratio bias. CONCLUSION: It is preferable to scan microarray slides at fixed, optimal gain settings under which the linearity between concentration and intensity is maximized. Although normalization methods improve reproducibility of microarray measurements, they appear less effective in improving accuracy.

Cross-platform comparability of microarray technology: intra-platform consistency and appropriate data analysis procedures are essential.

BACKGROUND: The acceptance of microarray technology in regulatory decision-making is being challenged by the existence of various platforms and data analysis methods. A recent report (E. Marshall, Science, 306, 630-631, 2004), by extensively citing the study of Tan et al. (Nucleic Acids Res., 31, 5676-5684, 2003), portrays a disturbingly negative picture of the cross-platform comparability, and, hence, the reliability of microarray technology. RESULTS: We reanalyzed Tan's dataset and found that the intra-platform consistency was low, indicating a problem in experimental procedures from which the dataset was generated. Furthermore, by using three gene selection methods (i.e., p-value ranking, fold-change ranking, and Significance Analysis of Microarrays (SAM)) on the same dataset we found that p-value ranking (the method emphasized by Tan et al.) results in much lower cross-platform concordance compared to fold-change ranking or SAM. Therefore, the low cross-platform concordance reported in Tan's study appears to be mainly due to a combination of low intra-platform consistency and a poor choice of data analysis procedures, instead of inherent technical differences among different platforms, as suggested by Tan et al. and Marshall. CONCLUSION: Our results illustrate the importance of establishing calibrated RNA samples and reference datasets to objectively assess the performance of different microarray platforms and the proficiency of individual laboratories as well as the merits of various data analysis procedures. Thus, we are progressively coordinating the MAQC project, a community-wide effort for microarray quality control.

Quantitative gene expression analysis in a nonhuman primate model of antibiotic-induced nephrotoxicity.

Gene expression patterns using microarrays have been described for rodent models of nephrotoxicity. To determine if significant gene expression changes previously identified have application across multiple species, we studied quantitative gene expression changes in the kidneys of female cynomolgus monkeys after exposure to two nephrotoxicants. Animals were dosed with the aminoglycoside gentamicin (10 mg/kg), the experimental oligosaccharide antibiotic everninomicin (30 or 60 mg/kg), or a combination of gentamicin (10 mg/kg) and everninomicin (30 mg/kg) for 7 days. Monkeys receiving these drugs in combination developed renal lesions as early as Day 1. By Day 7, monkeys dosed with 60 mg/kg everninomicin alone also developed renal lesions, while the group exposed to both compounds had more extensive renal damage. The modulation of several genes previously reported to be associated with nephrotoxicity in rodent models was confirmed using quantitative real-time PCR. Among these, waf-1, matrix metalloproteinase-9, and vimentin exhibited changes consistent with the definition of a genomic indicator of toxicity. In addition, we identified three early gene biomarkers that may be predictive of drug-induced nephrotoxicity: clusterin, osteopontin, and hepatitis A virus cellular receptor-1. Logistic regression demonstrated a high degree of correlation between changes in gene expression and the probability of the development of histopathologic lesions. These results are the first confirming rodent gene expression changes associated with nephrotoxicity in a nonhuman primate model and provide preliminary evidence for identifying early gene expression changes predicting the onset of drug-induced renal tubular damage in cynomolgus monkeys.

Identification and measurement of genomic biomarkers of nephrotoxicity.

An accelerated pace for drug discovery and development requires the application of measurements of novel parameters for drug-induced toxicity. Genomic biomarkers can contribute to this process as indicators or predictors of toxicity. Transcript levels for genes such as the kidney injury molecule have been identified and evaluated as genomic biomarkers of nephrotoxicity.

Quantitative PCR deconstruction of discrepancies between results reported by different hybridization platforms.

Differences in hybridization platforms used in gene array analysis experiments can lead to significant differences in hybridization results. In this study we used quantitative reverse transcription-polymerase chain reaction (qRT-PCR) to investigate discrepant results between the National Institute of Environmental Health Sciences cDNA and Affymetrix oligo platforms used to evaluate hepatic gene expression changes in rats exposed to methapyrilene. Caldesmon cDNA platform hybridization results showed decreases in gene expression levels for the high-dose methapyrilene 7-day pooled samples compared with their controls. By contrast, the Affymetrix oligonucleotide platform showed increases in expression levels for these samples. Quantitative gene expression measurements provide an explanation for the discrepancies observed for these samples. In the case of caldesmon, there is a 74-base sequence in the cDNA clone that is absent in the Affymetrix sequence. The amplicon based on the cDNA clone shows > 100-fold suppression relative to the day 7 high-dose methapyrilene-pooled control. These data demonstrate the importance of using a "gold standard," such as qRT-PCR to confirm key hybridization results as well as to understand the sources of discrepancies resulting from different hybridization platforms.