Two approaches for estimating the lower limit of quantitation (LLOQ) of microRNA levels assayed as exploratory biomarkers by RT-qPCR.

BACKGROUND: Circulating microRNAs are undergoing exploratory use as safety biomarkers in drug development. Reverse transcription quantitative polymerase chain reaction (RT-qPCR) is one common approach used to quantitate levels of microRNAs in samples that includes the use of a standard curve of calibrators fit to a regression model. Guidelines are needed for setting assay quantitation thresholds that are appropriate for this method and to biomarker pre-validation. RESULTS: In this report, we develop two workflows for determining a lower limit of quantitation (LLOQ) for RT-qPCR assays of microRNAs in exploratory studies. One workflow is based on an error threshold calculated by a logistic model of the calibration curve data. The second workflow is based on a threshold set by the sample blank, which is the no template control for RT-qPCR. The two workflows are used to set lower thresholds of reportable microRNA levels for an example dataset in which miR-208a levels in biofluids are quantitated in a cardiac injury model. LLOQ thresholds set by either workflow are effective in filtering out microRNA values with large uncertainty estimates. CONCLUSIONS: Two workflows for LLOQ determinations are presented in this report that provide methods that are easy to implement in investigational studies of microRNA safety biomarkers and offer choices in levels of conservatism in setting lower limits of acceptable values that facilitate interpretation of results.

Automatic text classification of drug-induced liver injury using document-term matrix and XGBoost.

Introduction: Regulatory agencies generate a vast amount of textual data in the review process. For example, drug labeling serves as a valuable resource for regulatory agencies, such as U.S. Food and Drug Administration (FDA) and Europe Medical Agency (EMA), to communicate drug safety and effectiveness information to healthcare professionals and patients. Drug labeling also serves as a resource for pharmacovigilance and drug safety research. Automated text classification would significantly improve the analysis of drug labeling documents and conserve reviewer resources. Methods: We utilized artificial intelligence in this study to classify drug-induced liver injury (DILI)-related content from drug labeling documents based on FDA's DILIrank dataset. We employed text mining and XGBoost models and utilized the Preferred Terms of Medical queries for adverse event standards to simplify the elimination of common words and phrases while retaining medical standard terms for FDA and EMA drug label datasets. Then, we constructed a document term matrix using weights computed by Term Frequency-Inverse Document Frequency (TF-IDF) for each included word/term/token. Results: The automatic text classification model exhibited robust performance in predicting DILI, achieving cross-validation AUC scores exceeding 0.90 for both drug labels from FDA and EMA and literature abstracts from the Critical Assessment of Massive Data Analysis (CAMDA). Discussion: Moreover, the text mining and XGBoost functions demonstrated in this study can be applied to other text processing and classification tasks.

Summarizing the incidence of adverse events using volcano plots and time intervals.

BACKGROUND: Adverse event incidence analyses are a critical component for describing the safety profile of any new intervention. The results typically are presented in lengthy summary tables. For therapeutic areas where patients have frequent adverse events, analysis and interpretation are made more difficult by the sheer number and variety of events that occur. Understanding the risk in these instances becomes even more crucial. PURPOSE: We describe a space-saving graphical summary that overcomes the limitations of traditional presentations of adverse events and improves interpretability of the safety profile. METHODS: We present incidence analyses of adverse events graphically using volcano plots to highlight treatment differences. Data from a clinical trial of patients experiencing an aneurysmal subarachnoid hemorrhage are used for illustration. Adjustments for multiplicity are illustrated. RESULTS: Color is used to indicate the treatment with higher incidence; bubble size represents the total number of events that occur in the treatment arms combined. Adjustments for multiple comparisons are displayed in a manner to indicate clearly those events for which the difference between treatment arms is statistically significant. Furthermore, adverse events can be displayed by time intervals, with multiple volcano plots or animation to appreciate changes in adverse event risk over time. Such presentations can emphasize early differences across treatments that may resolve later or highlight events for which treatment differences may become more substantial with longer follow-up. LIMITATIONS: Treatment arms are compared in a pairwise fashion. CONCLUSIONS: Volcano plots are space-saving tools that emphasize important differences between the adverse event profiles of two treatment arms. They can incorporate multiplicity adjustments in a manner that is straightforward to interpret and, by using time intervals, can illustrate how adverse event risk changes over the course of a clinical trial.

Anti-Oligomannose Antibodies as Potential Serum Biomarkers of Aggressive Prostate Cancer.

[Table: see text] This study bridges a carbohydrate microarray discovery and a large-scale serological validation of anti-oligomannose antibodies as novel serum biomarkers of aggressive prostate cancer (PCa). Experimentally, a Man9-cluster-specific enzyme-linked immunosorbent assay was established to enable sensitive detection of anti-Man9 antibodies in human sera. A large-cohort of men with PCa or benign prostatic hyperplasia (BPH) whose sera were banked at Stanford University was characterized using this assay. Subjects included patients with 100% Gleason grade 3 cancer (n = 84), with Gleason grades 4 and/or 5 cancer (n = 204), and BPH controls (n = 135). Radical prostatectomy Gleason grades and biochemical (PSA) recurrence served as key parameters for serum biomarker evaluation. It was found that IgGMan9 and IgMMan9 were widely present in the sera of men with BPH, as well as those with cancer. However, these antibody reactivities were significantly increased in the subjects with the largest volumes of high grade cancer. Detection of serum IgGMan9 and IgMMan9 significantly predicted the clinical outcome of PCa post-radical prostatectomy. Given these results, we suggest that IgGMan9 and IgMMan9 are novel serum biomarkers for monitoring aggressive progression of PCa. The potential of oligomannosyl antigens as targets for PCa subtyping and targeted immunotherapy is yet to be explored.

An efficient hierarchical generalized linear mixed model for pathway analysis of genome-wide association studies.

MOTIVATION: In genome-wide association studies (GWAS) of complex diseases, genetic variants having real but weak associations often fail to be detected at the stringent genome-wide significance level. Pathway analysis, which tests disease association with combined association signals from a group of variants in the same pathway, has become increasingly popular. However, because of the complexities in genetic data and the large sample sizes in typical GWAS, pathway analysis remains to be challenging. We propose a new statistical model for pathway analysis of GWAS. This model includes a fixed effects component that models mean disease association for a group of genes, and a random effects component that models how each gene's association with disease varies about the gene group mean, thus belongs to the class of mixed effects models. RESULTS: The proposed model is computationally efficient and uses only summary statistics. In addition, it corrects for the presence of overlapping genes and linkage disequilibrium (LD). Via simulated and real GWAS data, we showed our model improved power over currently available pathway analysis methods while preserving type I error rate. Furthermore, using the WTCCC Type 1 Diabetes (T1D) dataset, we demonstrated mixed model analysis identified meaningful biological processes that agreed well with previous reports on T1D. Therefore, the proposed methodology provides an efficient statistical modeling framework for systems analysis of GWAS. AVAILABILITY: The software code for mixed models analysis is freely available at http://biostat.mc.vanderbilt.edu/LilyWang.

Gene set analysis of genome-wide association studies: methodological issues and perspectives.

Recent studies have demonstrated that gene set analysis, which tests disease association with genetic variants in a group of functionally related genes, is a promising approach for analyzing and interpreting genome-wide association studies (GWAS) data. These approaches aim to increase power by combining association signals from multiple genes in the same gene set. In addition, gene set analysis can also shed more light on the biological processes underlying complex diseases. However, current approaches for gene set analysis are still in an early stage of development in that analysis results are often prone to sources of bias, including gene set size and gene length, linkage disequilibrium patterns and the presence of overlapping genes. In this paper, we provide an in-depth review of the gene set analysis procedures, along with parameter choices and the particular methodology challenges at each stage. In addition to providing a survey of recently developed tools, we also classify the analysis methods into larger categories and discuss their strengths and limitations. In the last section, we outline several important areas for improving the analytical strategies in gene set analysis.

Kernel density weighted loess normalization improves the performance of detection within asymmetrical data.

BACKGROUND: Normalization of gene expression data has been studied for many years and various strategies have been formulated to deal with various types of data. Most normalization algorithms rely on the assumption that the number of up-regulated genes and the number of down-regulated genes are roughly the same. However, the well-known Golden Spike experiment presents a unique situation in which differentially regulated genes are biased toward one direction, thereby challenging the conclusions of previous bench mark studies. RESULTS: This study proposes two novel approaches, KDL and KDQ, based on kernel density estimation to improve upon the basic idea of invariant set selection. The key concept is to provide various importance scores to data points on the MA plot according to their proximity to the cluster of the null genes under the assumption that null genes are more densely distributed than those that are differentially regulated. The comparison is demonstrated in the Golden Spike experiment as well as with simulation data using the ROC curves and compression rates. KDL and KDQ in combination with GCRMA provided the best performance among all approaches. CONCLUSIONS: This study determined that methods based on invariant sets are better able to resolve the problem of asymmetry. Normalization, either before or after expression summary for probesets, improves performance to a similar degree.

An integrated approach for the analysis of biological pathways using mixed models.

Gene class, ontology, or pathway testing analysis has become increasingly popular in microarray data analysis. Such approaches allow the integration of gene annotation databases, such as Gene Ontology and KEGG Pathway, to formally test for subtle but coordinated changes at a system level. Higher power in gene class testing is gained by combining weak signals from a number of individual genes in each pathway. We propose an alternative approach for gene-class testing based on mixed models, a class of statistical models that: a) provides the ability to model and borrow strength across genes that are both up and down in a pathway, b) operates within a well-established statistical framework amenable to direct control of false positive or false discovery rates, c) exhibits improved power over widely used methods under normal location-based alternative hypotheses, and d) handles complex experimental designs for which permutation resampling is difficult. We compare the properties of this mixed models approach with nonparametric method GSEA and parametric method PAGE using a simulation study, and illustrate its application with a diabetes data set and a dose-response data set.

Development and application of bovine and porcine oligonucleotide arrays with protein-based annotation.

The design of oligonucleotide sequences for the detection of gene expression in species with disparate volumes of genome and EST sequence information has been broadly studied. However, a congruous strategy has yet to emerge to allow the design of sensitive and specific gene expression detection probes. This study explores the use of a phylogenomic approach to align transcribed sequences to vertebrate protein sequences for the detection of gene families to design genomewide 70-mer oligonucleotide probe sequences for bovine and porcine. The bovine array contains 23,580 probes that target the transcripts of 16,341 genes, about 72% of the total number of bovine genes. The porcine array contains 19,980 probes targeting 15,204 genes, about 76% of the genes in the Ensembl annotation of the pig genome. An initial experiment using the bovine array demonstrates the specificity and sensitivity of the array.

A unified mixed effects model for gene set analysis of time course microarray experiments.

Methods for gene set analysis test for coordinated changes of a group of genes involved in the same biological process or molecular pathway. Higher statistical power is gained for gene set analysis by combining weak signals from a number of individual genes in each group. Although many gene set analysis methods have been proposed for microarray experiments with two groups, few can be applied to time course experiments. We propose a unified statistical model for analyzing time course experiments at the gene set level using random coefficient models, which fall into the more general class of mixed effects models. These models include a systematic component that models the mean trajectory for the group of genes, and a random component (the random coefficients) that models how each gene's trajectory varies about the mean trajectory. We show that the proposed model (1) outperforms currently available methods at discriminating gene sets differentially changed over time from null gene sets; (2) provides more stable results that are less affected by sampling variations; (3) models dependency among genes adequately and preserves type I error rate; and (4) allows for gene ranking based on predicted values of the random effects. We describe simulation studies using gene expression data with "real life" correlations and we demonstrate the proposed random coefficient model using a mouse colon development time course dataset. The agreement between results of the proposed random coefficient model and the previous reports for this proof-of-concept trial further validates this methodology, which provides a unified statistical model for systems analysis of microarray experiments with complex experimental designs when re-sampling based methods are difficult to apply.

Evaluation of external RNA controls for the assessment of microarray performance.

External RNA controls (ERCs), although important for microarray assay performance assessment, have yet to be fully implemented in the research community. As part of the MicroArray Quality Control (MAQC) study, two types of ERCs were implemented and evaluated; one was added to the total RNA in the samples before amplification and labeling; the other was added to the copyRNAs (cRNAs) before hybridization. ERC concentration-response curves were used across multiple commercial microarray platforms to identify problematic assays and potential sources of variation in the analytical process. In addition, the behavior of different ERC types was investigated, resulting in several important observations, such as the sample-dependent attributes of performance and the potential of using these control RNAs in a combinatorial fashion. This multiplatform investigation of the behavior and utility of ERCs provides a basis for articulating specific recommendations for their future use in evaluating assay performance across multiple platforms.

Performance comparison of one-color and two-color platforms within the MicroArray Quality Control (MAQC) project.

Microarray-based expression profiling experiments typically use either a one-color or a two-color design to measure mRNA abundance. The validity of each approach has been amply demonstrated. Here we provide a simultaneous comparison of results from one- and two-color labeling designs, using two independent RNA samples from the Microarray Quality Control (MAQC) project, tested on each of three different microarray platforms. The data were evaluated in terms of reproducibility, specificity, sensitivity and accuracy to determine if the two approaches provide comparable results. For each of the three microarray platforms tested, the results show good agreement with high correlation coefficients and high concordance of differentially expressed gene lists within each platform. Cumulatively, these comparisons indicate that data quality is essentially equivalent between the one- and two-color approaches and strongly suggest that this variable need not be a primary factor in decisions regarding experimental microarray design.

The balance of reproducibility, sensitivity, and specificity of lists of differentially expressed genes in microarray studies.

BACKGROUND: Reproducibility is a fundamental requirement in scientific experiments. Some recent publications have claimed that microarrays are unreliable because lists of differentially expressed genes (DEGs) are not reproducible in similar experiments. Meanwhile, new statistical methods for identifying DEGs continue to appear in the scientific literature. The resultant variety of existing and emerging methods exacerbates confusion and continuing debate in the microarray community on the appropriate choice of methods for identifying reliable DEG lists. RESULTS: Using the data sets generated by the MicroArray Quality Control (MAQC) project, we investigated the impact on the reproducibility of DEG lists of a few widely used gene selection procedures. We present comprehensive results from inter-site comparisons using the same microarray platform, cross-platform comparisons using multiple microarray platforms, and comparisons between microarray results and those from TaqMan - the widely regarded "standard" gene expression platform. Our results demonstrate that (1) previously reported discordance between DEG lists could simply result from ranking and selecting DEGs solely by statistical significance (P) derived from widely used simple t-tests; (2) when fold change (FC) is used as the ranking criterion with a non-stringent P-value cutoff filtering, the DEG lists become much more reproducible, especially when fewer genes are selected as differentially expressed, as is the case in most microarray studies; and (3) the instability of short DEG lists solely based on P-value ranking is an expected mathematical consequence of the high variability of the t-values; the more stringent the P-value threshold, the less reproducible the DEG list is. These observations are also consistent with results from extensive simulation calculations. CONCLUSION: We recommend the use of FC-ranking plus a non-stringent P cutoff as a straightforward and baseline practice in order to generate more reproducible DEG lists. Specifically, the P-value cutoff should not be stringent (too small) and FC should be as large as possible. Our results provide practical guidance to choose the appropriate FC and P-value cutoffs when selecting a given number of DEGs. The FC criterion enhances reproducibility, whereas the P criterion balances sensitivity and specificity.

Sources of variation in baseline gene expression levels from toxicogenomics study control animals across multiple laboratories.

BACKGROUND: The use of gene expression profiling in both clinical and laboratory settings would be enhanced by better characterization of variance due to individual, environmental, and technical factors. Meta-analysis of microarray data from untreated or vehicle-treated animals within the control arm of toxicogenomics studies could yield useful information on baseline fluctuations in gene expression, although control animal data has not been available on a scale and in a form best served for data-mining. RESULTS: A dataset of control animal microarray expression data was assembled by a working group of the Health and Environmental Sciences Institute's Technical Committee on the Application of Genomics in Mechanism Based Risk Assessment in order to provide a public resource for assessments of variability in baseline gene expression. Data from over 500 Affymetrix microarrays from control rat liver and kidney were collected from 16 different institutions. Thirty-five biological and technical factors were obtained for each animal, describing a wide range of study characteristics, and a subset were evaluated in detail for their contribution to total variability using multivariate statistical and graphical techniques. CONCLUSION: The study factors that emerged as key sources of variability included gender, organ section, strain, and fasting state. These and other study factors were identified as key descriptors that should be included in the minimal information about a toxicogenomics study needed for interpretation of results by an independent source. Genes that are the most and least variable, gender-selective, or altered by fasting were also identified and functionally categorized. Better characterization of gene expression variability in control animals will aid in the design of toxicogenomics studies and in the interpretation of their results.

An R2 statistic for fixed effects in the linear mixed model.

Statisticians most often use the linear mixed model to analyze Gaussian longitudinal data. The value and familiarity of the R(2) statistic in the linear univariate model naturally creates great interest in extending it to the linear mixed model. We define and describe how to compute a model R(2) statistic for the linear mixed model by using only a single model. The proposed R(2) statistic measures multivariate association between the repeated outcomes and the fixed effects in the linear mixed model. The R(2) statistic arises as a 1-1 function of an appropriate F statistic for testing all fixed effects (except typically the intercept) in a full model. The statistic compares the full model with a null model with all fixed effects deleted (except typically the intercept) while retaining exactly the same covariance structure. Furthermore, the R(2) statistic leads immediately to a natural definition of a partial R(2) statistic. A mixed model in which ethnicity gives a very small p-value as a longitudinal predictor of blood pressure (BP) compellingly illustrates the value of the statistic. In sharp contrast to the extreme p-value, a very small R(2) , a measure of statistical and scientific importance, indicates that ethnicity has an almost negligible association with the repeated BP outcomes for the study.

Breeding-assisted genomics: Applying meta-GWAS for milling and baking quality in CIMMYT wheat breeding program.

One of the biggest challenges for genetic studies on natural or unstructured populations is the unbalanced datasets where individuals are measured at different times and environments. This problem is also common in crop and animal breeding where many individuals are only evaluated for a single year and large but unbalanced datasets can be generated over multiple years. Many wheat breeding programs have focused on increasing bread wheat (Triticum aestivum L.) yield, but processing and end-use quality are critical components when considering its use in feeding the rising population of the next century. The challenges with end-use quality trait improvements are high cost and seed amounts for testing, the latter making selection in early breeding populations impossible. Here we describe a novel approach to identify marker-trait associations within a breeding program using a meta-genome wide association study (GWAS), which combines GWAS analysis from multi-year unbalanced breeding nurseries, in a manner reflecting meta-GWAS in humans. This method facilitated mapping of processing and end-use quality phenotypes from advanced breeding lines (n = 4,095) of the CIMMYT bread wheat breeding program from 2009 to 2014. Using the meta-GWAS we identified marker-trait associations, allele effects, candidate genes, and can select using markers generated in this process. Finally, the scope of this approach can be broadly applied in 'breeding-assisted genomics' across many crops to greatly increase our functional understanding of plant genomes.

A comparison of transcriptomic and metabonomic technologies for identifying biomarkers predictive of two-year rodent cancer bioassays.

Two-year rodent bioassays play a central role in evaluating the carcinogenic potential of both commercial products and environmental contaminants. The bioassays are expensive and time consuming, requiring years to complete and costing $2-4 million. In this study, we compare transcriptomic and metabonomic technologies for discovering biomarkers that can efficiently and economically identify chemical carcinogens without performing a standard two-year rodent bioassay. Animals were exposed subchronically to two chemicals (one genotoxic and one nongenotoxic) that were positive for lung and liver tumors in a standard two-year bioassay, two chemicals that were negative, and two control groups. Microarray analysis performed on liver and lung tissues identified multiple biomarkers in each tissue that could discriminate between carcinogenic and noncarcinogenic treatments. The discriminating biomarkers shared a common expression profile among carcinogenic treatments despite different genotoxicity categories and potential modes of action, suggesting that they reflect underlying cellular changes in the transition toward neoplasia. Statistical classification analysis exhibited 100% accuracy in both tissues when the number of genes was less than 5000. Additional genes reduced the predictive accuracy of the model. Serum samples were analyzed by 1H nuclear magnetic resonance (NMR) spectroscopy, and chemical-specific metabolites were removed from the spectra. The statistical classification analysis of the endogenous serum metabolites showed relatively low predictive accuracy with few metabolites in the model, but the accuracy increased to a maximum of 94% when all metabolites were added. These results suggest that individual endogenous metabolites are relatively poor biomarkers, but the metabolite profile as a whole is altered following carcinogen treatment.

Identification of dietary patterns associated with obesity in a nationally representative survey of Canadian adults: application of a priori, hybrid, and simplified dietary pattern techniques.

Background: Analyzing the effects of dietary patterns is an important approach for examining the complex role of nutrition in the etiology of obesity and chronic diseases.Objectives: The objectives of this study were to characterize the dietary patterns of Canadians with the use of a priori, hybrid, and simplified dietary pattern techniques, and to compare the associations of these patterns with obesity risk in individuals with and without chronic diseases (unhealthy and healthy obesity).Design: Dietary recalls from 11,748 participants (≥18 y of age) in the cross-sectional, nationally representative Canadian Community Health Survey 2.2 were used. A priori dietary pattern was characterized with the use of the previously validated 2015 Dietary Guidelines for Americans Adherence Index (DGAI). Weighted partial least squares (hybrid method) was used to derive an energy-dense (ED), high-fat (HF), low-fiber density (LFD) dietary pattern with the use of 38 food groups. The associations of derived dietary patterns with disease outcomes were then tested with the use of multinomial logistic regression.Results: An ED, HF, and LFD dietary pattern had high positive loadings for fast foods, carbonated drinks, and refined grains, and high negative loadings for whole fruits and vegetables (≥|0.17|). Food groups with a high loading were summed to form a simplified dietary pattern score. Moving from the first (healthiest) to the fourth (least healthy) quartiles of the ED, HF, and LFD pattern and the simplified dietary pattern scores was associated with increasingly elevated ORs for unhealthy obesity, with individuals in quartile 4 having an OR of 2.57 (95% CI: 1.75, 3.76) and 2.73 (95% CI: 1.88, 3.98), respectively (P-trend < 0.0001). Individuals who adhered the most to the 2015 DGAI recommendations (quartile 4) had a 53% lower OR of unhealthy obesity (P-trend < 0.0001). The associations of dietary patterns with healthy obesity and unhealthy nonobesity were weaker, albeit significant.Conclusions: Consuming an ED, HF, and LFD dietary pattern and lack of adherence to the recommendations of the 2015 DGAI were associated with a significantly higher risk of obesity with and without accompanying chronic diseases.

A DREAM Challenge to Build Prediction Models for Short-Term Discontinuation of Docetaxel in Metastatic Castration-Resistant Prostate Cancer.

PURPOSE: Docetaxel has a demonstrated survival benefit for patients with metastatic castration-resistant prostate cancer (mCRPC); however, 10% to 20% of patients discontinue docetaxel prematurely because of toxicity-induced adverse events, and the management of risk factors for toxicity remains a challenge. PATIENTS AND METHODS: The comparator arms of four phase III clinical trials in first-line mCRPC were collected, annotated, and compiled, with a total of 2,070 patients. Early discontinuation was defined as treatment stoppage within 3 months as a result of adverse treatment effects; 10% of patients discontinued treatment. We designed an open-data, crowd-sourced DREAM Challenge for developing models with which to predict early discontinuation of docetaxel treatment. Clinical features for all four trials and outcomes for three of the four trials were made publicly available, with the outcomes of the fourth trial held back for unbiased model evaluation. Challenge participants from around the world trained models and submitted their predictions. Area under the precision-recall curve was the primary metric used for performance assessment. RESULTS: In total, 34 separate teams submitted predictions. Seven models with statistically similar area under precision-recall curves (Bayes factor ≤ 3) outperformed all other models. A postchallenge analysis of risk prediction using these seven models revealed three patient subgroups: high risk, low risk, or discordant risk. Early discontinuation events were two times higher in the high-risk subgroup compared with the low-risk subgroup. Simulation studies demonstrated that use of patient discontinuation prediction models could reduce patient enrollment in clinical trials without the loss of statistical power. CONCLUSION: This work represents a successful collaboration between 34 international teams that leveraged open clinical trial data. Our results demonstrate that routinely collected clinical features can be used to identify patients with mCRPC who are likely to discontinue treatment because of adverse events and establishes a robust benchmark with implications for clinical trial design.