Seeking the wisdom of crowds through challenge-based competitions in biomedical research.

"Challenge-based competitions" refers to a framework for addressing fundamental research questions in which the community is presented with a challenge, the data to address the challenge, and independent, unbiased assessment to rank submitted solutions. Although the typical result of such efforts is a robust performance evaluation of diverse methodologies, challenge-based competitions reach far beyond algorithm assessment. Here, we discuss the impact of challenge-based competitions in the areas of organizing and building communities and driving innovation.

Statistical analysis of MPSS measurements: application to the study of LPS-activated macrophage gene expression.

Massively Parallel Signature Sequencing (MPSS), a recently developed high-throughput transcription profiling technology, has the ability to profile almost every transcript in a sample without requiring prior knowledge of the sequence of the transcribed genes. As is the case with DNA microarrays, effective data analysis depends crucially on understanding how noise affects measurements. We analyze the sources of noise in MPSS and present a quantitative model describing the variability between replicate MPSS assays. We use this model to construct statistical hypotheses that test whether an observed change in gene expression in a pair-wise comparison is significant. This analysis is then extended to the determination of the significance of changes in expression levels measured over the course of a time series of measurements. We apply these analytic techniques to the study of a time series of MPSS gene expression measurements on LPS-stimulated macrophages. To evaluate our statistical significance metrics, we compare our results with published data on macrophage activation measured by using Affymetrix GeneChips.

p53-Mdm2 loop controlled by a balance of its feedback strength and effective dampening using ATM and delayed feedback.

When the genomic integrity of a cell is challenged, its fate is determined in part by signals conveyed by the p53 tumour suppressor protein. It was observed recently that such signals are not simple gradations of p53 concentration, but rather a counter-intuitive limit-cycle behaviour. Based on a careful mathematical interpretation of the experimental body of knowledge, we propose a model for the p53 signalling network and characterise the p53 stability and oscillatory dynamics. In our model, ATM, a protein that senses DNA damage, activates p53 by phosphorylation. In its active state, p53 has a decreased degradation rate and an enhanced transactivation of Mdm2, a gene whose protein product Mdm2 tags p53 for degradation. Thus the p53-Mdm2 system forms a negative feedback loop. However, the feedback in this loop is delayed, as the pool of Mdm2 molecules being induced by p53 at a given time will mark for degradation the pool of p53 molecules at some later time, after the Mdm2 molecules have been transcribed, exported out of the nucleus, translated and transported back into the nucleus. The analysis of our model demonstrates how this time lag combines with the ATM-controlled feedback strength and effective dampening of the negative feedback loop to produce limit-cycle oscillations. The picture that emerges is that ATM, once activated by DNA damage, makes the p53-Mdm2 oscillator undergo a supercritical Hopf bifurcation. This approach yields an improved understanding of the global dynamics and bifurcation structure of our time-delayed, negative feedback model and allows for predictions of the behaviour of the p53 system under different perturbations.

Quantitative noise analysis for gene expression microarray experiments.

A major challenge in DNA microarray analysis is to effectively dissociate actual gene expression values from experimental noise. We report here a detailed noise analysis for oligonuleotide-based microarray experiments involving reverse transcription, generation of labeled cRNA (target) through in vitro transcription, and hybridization of the target to the probe immobilized on the substrate. By designing sets of replicate experiments that bifurcate at different steps of the assay, we are able to separate the noise caused by sample preparation and the hybridization processes. We quantitatively characterize the strength of these different sources of noise and their respective dependence on the gene expression level. We find that the sample preparation noise is small, implying that the amplification process during the sample preparation is relatively accurate. The hybridization noise is found to have very strong dependence on the expression level, with different characteristics for the low and high expression values. The hybridization noise characteristics at the high expression regime are mostly Poisson-like, whereas its characteristics for the small expression levels are more complex, probably due to cross-hybridization. A method to evaluate the significance of gene expression fold changes based on noise characteristics is proposed.

Gene expression profiling of B cell chronic lymphocytic leukemia reveals a homogeneous phenotype related to memory B cells.

B cell-derived chronic lymphocytic leukemia (B-CLL) represents a common malignancy whose cell derivation and pathogenesis are unknown. Recent studies have shown that >50% of CLLs display hypermutated immunoglobulin variable region (IgV) sequences and a more favorable prognosis, suggesting that they may represent a distinct subset of CLLs which have transited through germinal centers (GCs), the physiologic site of IgV hypermutation. To further investigate the phenotype of CLLs, their cellular derivation and their relationship to normal B cells, we have analyzed their gene expression profiles using oligonucleotide-based DNA chip microarrays representative of approximately 12,000 genes. The results show that CLLs display a common and characteristic gene expression profile that is largely independent of their IgV genotype. Nevertheless, a restricted number of genes (<30) have been identified whose differential expression can distinguish IgV mutated versus unmutated cases and identify them in independent panels of cases. Comparison of CLL profiles with those of purified normal B cell subpopulations indicates that the common CLL profile is more related to memory B cells than to those derived from naive B cells, CD5(+) B cells, and GC centroblasts and centrocytes. Finally, this analysis has identified a subset of genes specifically expressed by CLL cells of potential pathogenetic and clinical relevance.

Systematic and fully automated identification of protein sequence patterns.

We present an efficient algorithm to systematically and automatically identify patterns in protein sequence families. The procedure is based on the Splash deterministic pattern discovery algorithm and on a framework to assess the statistical significance of patterns. We demonstrate its application to the fully automated discovery of patterns in 974 PROSITE families (the complete subset of PROSITE families which are defined by patterns and contain DR records). Splash generates patterns with better specificity and undiminished sensitivity, or vice versa, in 28% of the families; identical statistics were obtained in 48% of the families, worse statistics in 15%, and mixed behavior in the remaining 9%. In about 75% of the cases, Splash patterns identify sequence sites that overlap more than 50% with the corresponding PROSITE pattern. The procedure is sufficiently rapid to enable its use for daily curation of existing motif and profile databases. Third, our results show that the statistical significance of discovered patterns correlates well with their biological significance. The trypsin subfamily of serine proteases is used to illustrate this method's ability to exhaustively discover all motifs in a family that are statistically and biologically significant. Finally, we discuss applications of sequence patterns to multiple sequence alignment and the training of more sensitive score-based motif models, akin to the procedure used by PSI-BLAST. All results are available at httpl//www.research.ibm.com/spat/.

Catalytic tempering: A method for sampling rough energy landscapes by Monte Carlo.

A new Monte Carlo algorithm is presented for the efficient sampling of the Boltzmann distribution of configurations of systems with rough energy landscapes. The method is based on the introduction of a fictitious coordinate y so that the dimensionality of the system is increased by one. This augmented system has a potential surface and a temperature that is made to depend on the new coordinate y in such a way that for a small strip of the y space, called the "normal region," the temperature is set equal to the temperature desired and the potential is the original rough energy potential. To enhance barrier crossing outside the "normal region," the energy barriers are reduced by truncation (with preservation of the potential minima) and the temperature is made to increase with ||y ||. The method, called catalytic tempering or CAT, is found to greatly improve the rate of convergence of Monte Carlo sampling in model systems and to eliminate the quasi-ergodic behavior often found in the sampling of rough energy landscapes.

Analysis of gene expression microarrays for phenotype classification.

Several microarray technologies that monitor the level of expression of a large number of genes have recently emerged. Given DNA-microarray data for a set of cells characterized by a given phenotype and for a set of control cells, an important problem is to identify "patterns" of gene expression that can be used to predict cell phenotype. The potential number of such patterns is exponential in the number of genes. In this paper, we propose a solution to this problem based on a supervised learning algorithm, which differs substantially from previous schemes. It couples a complex, non-linear similarity metric, which maximizes the probability of discovering discriminative gene expression patterns, and a pattern discovery algorithm called SPLASH. The latter discovers efficiently and deterministically all statistically significant gene expression patterns in the phenotype set. Statistical significance is evaluated based on the probability of a pattern to occur by chance in the control set. Finally, a greedy set covering algorithm is used to select an optimal subset of statistically significant patterns, which form the basis for a standard likelihood ratio classification scheme. We analyze data from 60 human cancer cell lines using this method, and compare our results with those of other supervised learning schemes. Different phenotypes are studied. These include cancer morphologies (such as melanoma), molecular targets (such as mutations in the p53 gene), and therapeutic targets related to the sensitivity to an anticancer compounds. We also analyze a synthetic data set that shows that this technique is especially well suited for the analysis of sub-phenotype mixtures. For complex phenotypes, such as p53, our method produces an encouragingly low rate of false positives and false negatives and seems to outperform the others. Similar low rates are reported when predicting the efficacy of experimental anticancer compounds. This counts among the first reported studies where drug efficacy has been successfully predicted from large-scale expression data analysis.

Compositional heterogeneity within, and uniformity between, DNA sequences of yeast chromosomes.

The heterogeneity within, and similarities between, yeast chromosomes are studied. For the former, we show by the size distribution of domains, coding density, size distribution of open reading frames, spatial power spectra, and deviation from binomial distribution for C + G% in large moving windows that there is a strong deviation of the yeast sequences from random sequences. For the latter, not only do we graphically illustrate the similarity for the above mentioned statistics, but we also carry out a rigorous analysis of variance (ANOVA) test. The hypothesis that all yeast chromosomes are similar cannot be rejected by this test. We examine the two possible explanations of this interchromosomal uniformity: a common origin, such as genome-wide duplication (polyploidization), and a concerted evolutionary process.

Mnemonics for variability: remembering food delay.

Three experiments with White Carneaux pigeons (Columba livia) investigated memory and decision processes under fixed and variable reinforcement intervals. Response rate was measured during the unreinforced trials in the discrete-trial peak procedure in which reinforced trials were mixed with long unreinforced trials. Two decision models differing in assumptions about memory constraints are reviewed. In the complete-memory model (J. Gibbon, R.M. Church, S. Fairhurst, & A. Kacelnik, 1988), all interreinforcement intervals were remembered, whereas in the minimax model (D. Brunner, A. Kacelnik, & J. Gibbon, 1996), only estimates of the shortest and longest possible reinforcement times were remembered. Both models accommodated some features of response rate as a function of trial time, but only the second was compatible with the observed cessation of responding.

Efficiency of DNA replication in the polymerase chain reaction.

A detailed quantitative kinetic model for the polymerase chain reaction (PCR) is developed, which allows us to predict the probability of replication of a DNA molecule in terms of the physical parameters involved in the system. The important issue of the determination of the number of PCR cycles during which this probability can be considered to be a constant is solved within the framework of the model. New phenomena of multimodality and scaling behavior in the distribution of the number of molecules after a given number of PCR cycles are presented. The relevance of the model for quantitative PCR is discussed, and a novel quantitative PCR technique is proposed.