The future of direct-to-consumer clinical genetic tests.

In light of the meeting of the US Food and Drug Administration (FDA) in March 2011 to discuss the regulation of clinical direct-to-consumer (DTC) genetic tests, we have invited five experts to consider the best means of overseeing the ordering and interpretation of these tests. Should these tests be regulated? If so, who, if anyone, should communicate results to consumers?

Regulation, reimbursement, and the long road of implementation of personalized medicine--a perspective from the United States.

There is undisputed evidence that personalized medicine, that is, a more precise assessment of which medical intervention might best serve an individual patient on the basis of novel technology, such as molecular profiling, can have a significant impact on clinical outcomes. The field, however, is still new, and the demonstration of improved effectiveness compared with standard of care comes at a cost. How can we be sure that personalized medicine indeed provides a measurable clinical benefit, that we will be able to afford it, and that we can provide adequate access? The risk-benefit evaluation that accompanies each medical decision requires not only good clinical data but also an assessment of cost and infrastructure needed to provide access to technology. Several examples from the last decade illustrate which types of personalized medicines and diagnostic tests are easily being taken up in clinical practice and which types are more difficult to introduce. And as regulators and payers in the United States and elsewhere are taking on personalized medicine, an interesting convergence can be observed: better, more complete information for both approval and coverage decisions could be gained from a coordination of regulatory and reimbursement questions. Health economics and outcomes research (HEOR) emerges as an approach that can satisfy both needs. Although HEOR represents a well-established approach to demonstrate the effectiveness of interventions in many areas of medical practice, few HEOR studies exist in the field of personalized medicine today. It is reasonable to expect that this will change over the next few years.

The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements.

Over the last decade, the introduction of microarray technology has had a profound impact on gene expression research. The publication of studies with dissimilar or altogether contradictory results, obtained using different microarray platforms to analyze identical RNA samples, has raised concerns about the reliability of this technology. The MicroArray Quality Control (MAQC) project was initiated to address these concerns, as well as other performance and data analysis issues. Expression data on four titration pools from two distinct reference RNA samples were generated at multiple test sites using a variety of microarray-based and alternative technology platforms. Here we describe the experimental design and probe mapping efforts behind the MAQC project. We show intraplatform consistency across test sites as well as a high level of interplatform concordance in terms of genes identified as differentially expressed. This study provides a resource that represents an important first step toward establishing a framework for the use of microarrays in clinical and regulatory settings.

The balance of reproducibility, sensitivity, and specificity of lists of differentially expressed genes in microarray studies.

BACKGROUND: Reproducibility is a fundamental requirement in scientific experiments. Some recent publications have claimed that microarrays are unreliable because lists of differentially expressed genes (DEGs) are not reproducible in similar experiments. Meanwhile, new statistical methods for identifying DEGs continue to appear in the scientific literature. The resultant variety of existing and emerging methods exacerbates confusion and continuing debate in the microarray community on the appropriate choice of methods for identifying reliable DEG lists. RESULTS: Using the data sets generated by the MicroArray Quality Control (MAQC) project, we investigated the impact on the reproducibility of DEG lists of a few widely used gene selection procedures. We present comprehensive results from inter-site comparisons using the same microarray platform, cross-platform comparisons using multiple microarray platforms, and comparisons between microarray results and those from TaqMan - the widely regarded "standard" gene expression platform. Our results demonstrate that (1) previously reported discordance between DEG lists could simply result from ranking and selecting DEGs solely by statistical significance (P) derived from widely used simple t-tests; (2) when fold change (FC) is used as the ranking criterion with a non-stringent P-value cutoff filtering, the DEG lists become much more reproducible, especially when fewer genes are selected as differentially expressed, as is the case in most microarray studies; and (3) the instability of short DEG lists solely based on P-value ranking is an expected mathematical consequence of the high variability of the t-values; the more stringent the P-value threshold, the less reproducible the DEG list is. These observations are also consistent with results from extensive simulation calculations. CONCLUSION: We recommend the use of FC-ranking plus a non-stringent P cutoff as a straightforward and baseline practice in order to generate more reproducible DEG lists. Specifically, the P-value cutoff should not be stringent (too small) and FC should be as large as possible. Our results provide practical guidance to choose the appropriate FC and P-value cutoffs when selecting a given number of DEGs. The FC criterion enhances reproducibility, whereas the P criterion balances sensitivity and specificity.

Qualification of biomarkers for drug development in organ transplantation.

The drug development process is dependent upon having established end points for measuring drug efficacy and adverse effects. New drug development in organ transplantation suffers from having end points which are either outdated or which do not serve the purpose of addressing the current critical drug therapy problems. Numerous biomarkers have been examined in organ transplantation, but almost all would be classified as exploratory for drug development purposes. Some of the possible pathways out of this dilemma include investigator- or consortium-initiated research that would qualify the biomarkers as either probable or known valid biomarkers, help in identification of new end points in transplantation and their associated biomarkers, co-development of a new biomarker and drug for transplantation and the use of new clinical trial design methods which facilitate enriched or stratified transplant patient populations. With new biomarkers and new study design methodologies for drug development, improvement in the drug development process for transplantation is a real possibility that the transplant clinical and research community can help to bring about.

Pharmacogenomic biomarker information in drug labels approved by the United States food and drug administration: prevalence of related drug use.

STUDY OBJECTIVES: To review the labels of United States Food and Drug Administration (FDA)-approved drugs to identify those that contain pharmacogenomic biomarker information, and to collect prevalence information on the use of those drugs for which pharmacogenomic information is included in the drug labeling. DESIGN: Retrospective analysis. DATA SOURCES: The Physicians' Desk Reference Web site, Drugs@FDA Web site, and manufacturers' Web sites were used to identify drug labels containing pharmacogenomic information, and the prescription claims database of a large pharmacy benefits manager (insuring > 55 million individuals in the United States) was used to obtain drug utilization data. MEASUREMENTS AND MAIN RESULTS: Pharmacogenomic biomarkers were defined, FDA-approved drug labels containing this information were identified, and utilization of these drugs was determined. Of 1200 drug labels reviewed for the years 1945-2005, 121 drug labels contained pharmacogenomic information based on a key word search and follow-up screening. Of those, 69 labels referred to human genomic biomarkers, and 52 referred to microbial genomic biomarkers. Of the labels referring to human biomarkers, 43 (62%) pertained to polymorphisms in cytochrome P450 (CYP) enzyme metabolism, with CYP2D6 being most common. Of 36.1 million patients whose prescriptions were processed by a large pharmacy benefits manager in 2006, about 8.8 million (24.3%) received one or more drugs with human genomic biomarker information in the drug label. CONCLUSION: Nearly one fourth of all outpatients received one or more drugs that have pharmacogenomic information in the label for that drug. The incorporation and appropriate use of pharmacogenomic information in drug labels should be tested for its ability to improve drug use and safety in the United States.

Strategic paths for biomarker qualification.

Biomarkers may be qualified using different qualification processes. A passive approach for qualification has been to accept the end of discussions in the scientific literature as an indication that a biomarker has been accepted. An active approach to qualification requires development of a comprehensive process by which a consensus may be reached about the qualification of a biomarker. Active strategies for qualification include those associated with context-independent as well as context-dependent qualifications.

Implementing the U.S. FDA guidance on pharmacogenomic data submissions.

The FDA Guidance for Industry: Pharmacogenomics Data Submissions was issued in 2005. This guidance document covers a broad area associated with how and when to submit genomic data to the FDA. Additional tasks associated with genomic data submissions include the implementation of genomic data submissions; the process for qualification of exploratory biomarkers into valid biomarkers; and technical recommendations for the generation and submission of genomic data to the FDA. These tasks have been addressed throughout the past 2 years by a number of initiatives. These initiatives have included the development of the Interdisciplinary Pharmacogenomics Review Group for review of pharmacogenomic data submissions, the pilot process for qualification of biomarkers, and the concept paper on recommendations for the generation and submission of genomic data. These initiatives have contributed to the effective implementation of the Pharmacogenomics Guidance at the FDA.

Questions and answers about the Pilot Process for Biomarker Qualification at the FDA.

The FDA has developed a Pilot Process for Biomarker Qualification. Initial experience with this process has underscored the care that a long-term approach to biomarker qualification independently of development for specific drugs should have in the assembly of external industry consortia as well as the internal regulatory organization for these qualification efforts. There are complex scientific and clinical issues associated with these qualifications, and it is paramount that the expertise needed for their review be identified so that a comprehensive consensus may be reached at the end of this process. Several frequently asked questions associated with this process are presented in this paper, as well as answers reflecting the Agency's current thinking about this process.:

The Predictive Safety Testing Consortium: A synthesis of the goals, challenges and accomplishments of the Critical Path.

The qualification of biomarkers of drug safety requires data on many compounds and nonclinical and clinical studies. The cost and effort associated with these qualifications cannot be easily covered by a single pharmaceutical company. Intellectual property associated with safety biomarkers is also held by many different companies. Consortia between different pharmaceutical companies can overcome cost and intellectual property hurdles to biomarker qualification. The Predictive Safety Testing Consortium (PSTC) is a collaborative effort between 16 different pharmaceutical companies to generate data supporting biomarker qualification. This Consortium is coordinated through the C-Path Institute, and currently has five biomarker qualification working groups engaged in this collaboration: nephrotoxicity, hepatotoxicity, vascular injury, myopathy, and non-genotoxic carcinogenicity. These working groups are aided by a data management team and a translational strategy team. Qualification studies of promising biomarkers are already progressing in several of the working groups, and results in the nephrotoxicity working group warranted a data submission to the FDA and EMEA for regulatory qualification of new nephrotoxicity biomarkers.:

An integrated bioinformatics infrastructure essential for advancing pharmacogenomics and personalized medicine in the context of the FDA's Critical Path Initiative.

Pharmacogenomics (PGx) is identified in the FDA Critical Path document as a major opportunity for advancing medical product development and personalized medicine. An integrated bioinformatics infrastructure for use in FDA data review is crucial to realize the benefits of PGx for public health. We have developed an integrated bioinformatics tool, called ArrayTrack, for managing, analyzing and interpreting genomic and other biomarker data (e.g. proteomic and metabolomic data). ArrayTrack is a highly flexible and robust software platform, which allows evolving with technological advances and changing user needs. ArrayTrack is used in the routine review of genomic data submitted to the FDA; here, three hypothetical examples of its use in the Voluntary eXploratory Data Submission (VXDS) program are illustrated.:

Microarray scanner calibration curves: characteristics and implications.

BACKGROUND: Microarray-based measurement of mRNA abundance assumes a linear relationship between the fluorescence intensity and the dye concentration. In reality, however, the calibration curve can be nonlinear. RESULTS: By scanning a microarray scanner calibration slide containing known concentrations of fluorescent dyes under 18 PMT gains, we were able to evaluate the differences in calibration characteristics of Cy5 and Cy3. First, the calibration curve for the same dye under the same PMT gain is nonlinear at both the high and low intensity ends. Second, the degree of nonlinearity of the calibration curve depends on the PMT gain. Third, the two PMTs (for Cy5 and Cy3) behave differently even under the same gain. Fourth, the background intensity for the Cy3 channel is higher than that for the Cy5 channel. The impact of such characteristics on the accuracy and reproducibility of measured mRNA abundance and the calculated ratios was demonstrated. Combined with simulation results, we provided explanations to the existence of ratio underestimation, intensity-dependence of ratio bias, and anti-correlation of ratios in dye-swap replicates. We further demonstrated that although Lowess normalization effectively eliminates the intensity-dependence of ratio bias, the systematic deviation from true ratios largely remained. A method of calculating ratios based on concentrations estimated from the calibration curves was proposed for correcting ratio bias. CONCLUSION: It is preferable to scan microarray slides at fixed, optimal gain settings under which the linearity between concentration and intensity is maximized. Although normalization methods improve reproducibility of microarray measurements, they appear less effective in improving accuracy.

Cross-platform comparability of microarray technology: intra-platform consistency and appropriate data analysis procedures are essential.

BACKGROUND: The acceptance of microarray technology in regulatory decision-making is being challenged by the existence of various platforms and data analysis methods. A recent report (E. Marshall, Science, 306, 630-631, 2004), by extensively citing the study of Tan et al. (Nucleic Acids Res., 31, 5676-5684, 2003), portrays a disturbingly negative picture of the cross-platform comparability, and, hence, the reliability of microarray technology. RESULTS: We reanalyzed Tan's dataset and found that the intra-platform consistency was low, indicating a problem in experimental procedures from which the dataset was generated. Furthermore, by using three gene selection methods (i.e., p-value ranking, fold-change ranking, and Significance Analysis of Microarrays (SAM)) on the same dataset we found that p-value ranking (the method emphasized by Tan et al.) results in much lower cross-platform concordance compared to fold-change ranking or SAM. Therefore, the low cross-platform concordance reported in Tan's study appears to be mainly due to a combination of low intra-platform consistency and a poor choice of data analysis procedures, instead of inherent technical differences among different platforms, as suggested by Tan et al. and Marshall. CONCLUSION: Our results illustrate the importance of establishing calibrated RNA samples and reference datasets to objectively assess the performance of different microarray platforms and the proficiency of individual laboratories as well as the merits of various data analysis procedures. Thus, we are progressively coordinating the MAQC project, a community-wide effort for microarray quality control.

From pharmacogenetics to personalized medicine: a vital need for educating health professionals and the community.

The field of pharmacogenetics will soon celebrate its 50th anniversary. Although science has delivered an impressive amount of information in these 50 years, pharmacogenetics has suffered from lack of integration into clinical practice. There are several reasons for this, including the unmet need for education at medical schools and the lack of awareness about the impact of genetic medicine on healthcare in the community. Recently, the FDA announced that it considers pharmacogenomics one of three major opportunities on the critical path to new medical products. This notion by the FDA is filling the regulatory void that existed between drug developers and drug users. However, in order to bring pharmacogenetic testing to the prescription pad successfully, healthcare professionals and policy makers, as well as patients, need to have the necessary background knowledge for making educated treatment decisions. To effectively move pharmacogenetics into everyday medicine, it is therefore imperative for scientists and teachers in the field to take on the challenge of disseminating pharmacogenetic insights to a broader audience.

QA/QC: challenges and pitfalls facing the microarray community and regulatory agencies.

The scientific community has been enthusiastic about DNA microarray technology for pharmacogenomic and toxicogenomic studies in the hope of advancing personalized medicine and drug development. The US Food and Drug Administration has been proactive in promoting the use of pharmacogenomic data in drug development and has issued a draft guidance for the pharmaceutical industry on data submissions. However, many challenges and pitfalls are facing the microarray community and regulatory agencies before microarray data can be reliably applied to support regulatory decision making. Four types of factors (i.e., technical, instrumental, computational and interpretative) affect the outcome of a microarray study, and a major concern about microarray studies has been the lack of reproducibility and accuracy. Intralaboratory data consistency is the foundation of reliable knowledge extraction and meaningful crosslaboratory or crossplatform comparisons; unfortunately, it has not been seriously evaluated and demonstrated in every study. Profound problems in data quality have been observed from analyzing published data sets, and many laboratories have been struggling with technical troubleshooting rather than generating reliable data of scientific significance. The microarray community and regulatory agencies must work together to establish a set of consensus quality assurance and quality control criteria for assessing and ensuring data quality, to identify critical factors affecting data quality, and to optimize and standardize microarray procedures so that biologic interpretation and decision-making are not based on unreliable data. These fundamental issues must be adequately addressed before microarray technology can be transformed from a research tool to clinical practices.

Molecular diagnostics clinical utility strategy: a six-part framework.

The clinical utility of a molecular test rises proportional to a favorable regulatory risk/benefit assessment, and clinical utility is the driver of payer coverage decisions. Although a great deal has been written about clinical utility, debates still center on its 'definition.' We argue that the definition (an impact on clinical outcomes) is self-evident, and improved communications should focus on sequential steps in building and proving an adequate level of confidence for the diagnostic test's clinical value proposition. We propose a six-part framework to facilitate communications between test developers and health technology evaluators, relevant to both regulatory and payer decisions.

Evidence of clinical utility: an unmet need in molecular diagnostics for patients with cancer.

This article defines and describes best practices for the academic and business community to generate evidence of clinical utility for cancer molecular diagnostic assays. Beyond analytical and clinical validation, successful demonstration of clinical utility involves developing sufficient evidence to demonstrate that a diagnostic test results in an improvement in patient outcomes. This discussion is complementary to theoretical frameworks described in previously published guidance and literature reports by the U.S. Food and Drug Administration, Centers for Disease Control and Prevention, Institute of Medicine, and Center for Medical Technology Policy, among others. These reports are comprehensive and specifically clarify appropriate clinical use, adoption, and payer reimbursement for assay manufacturers, as well as Clinical Laboratory Improvement Amendments-certified laboratories, including those that develop assays (laboratory developed tests). Practical criteria and steps for establishing clinical utility are crucial to subsequent decisions for reimbursement without which high-performing molecular diagnostics will have limited availability to patients with cancer and fail to translate scientific advances into high-quality and cost-effective cancer care. See all articles in this CCR Focus section, "The Precision Medicine Conundrum: Approaches to Companion Diagnostic Co-development."

Physician Awareness and Utilization of Food and Drug Administration (FDA)-Approved Labeling for Pharmacogenomic Testing Information.

We surveyed 10,303 United States physicians on where they obtain pharmacogenomic testing information. Thirty-nine percent indicated that they obtained this from drug labeling. Factors positively associated with this response included older age, postgraduate instruction, using other information sources, regulatory approval/ recommendation of testing, reliance on labeling for information, and perception that patients have benefited from testing. Physicians use pharmacogenomic testing information from drug labeling, highlighting the importance of labeling information that is conducive to practice application.

On rat poison and human medicines: personalizing warfarin therapy.

Teaching old dogs new tricks is difficult, but lessons learned from such efforts can be invaluable. Warfarin is an old drug, difficult to administer and a leading cause of drug-related mortality and hospitalizations. New genetic tests for optimizing warfarin therapy have not been adopted. The debate over precise clinical utility and cost-effectiveness of these tests misses more important points of building a better, cheaper, and more efficient infrastructure to measure the true real-world impact of personalized medicine. However, this same debate about how, when, and where such testing is appropriate has been invaluable to the field of personalized medicine: progress beyond science, in policy, regulations, and logistics can be highlighted along the path to safer and more efficacious warfarin therapy.