Application Health Monitoring for Extreme-scale Resiliency using Cooperative Fault Management.

Resiliency is and will be a critical factor in determining scientific productivity on current and exascale supercomputers, and beyond. Applications oblivious to and incapable of handling transient soft and hard errors could waste supercomputing resources or, worse, yield misleading scientific insights. We introduce a novel application-driven silent error detection and recovery strategy based on application health monitoring. Our methodology uses application output that follows known patterns as indicators of an application's health, and knowledge that violation of these patterns could be indication of faults. Information from system monitors that report hardware and software health status is used to corroborate faults. Collectively, this information is used by a fault coordinator agent to take preventive and corrective measures by applying computational steering to an application between checkpoints. This cooperative fault management system uses the Fault Tolerance Backplane as a communication channel. The benefits of this framework are demonstrated with two real application case studies, molecular dynamics and quantum chemistry simulations, on scalable clusters with simulated memory and I/O corruptions. The developed approach is general and can be easily applied to other applications.

An evolution-based analysis scheme to identify CO2/O2 specificity-determining factors for ribulose 1,5-bisphosphate carboxylase/oxygenase.

Ribulose 1,5-bisphosphate carboxylase/oxygenase (RuBisCo) catalyzes a rate-limiting step in photosynthetic carbon assimilation (reacting with CO2) and its competitive photo-respiratory carbon oxidation (reacting with O2). RuBisCo enzyme with an enhanced CO2/O2 specificity would boost the ability to make great progress in agricultural production and environmental management. RuBisCos in marine non-green algae, resulting from an earlier endo-symbiotic event, diverge greatly from those in green plants and cyanobacteria and, further, have the highest CO2/O2 specificity whereas RuBisCos in cyanobacteria have the lowest. We assumed that there exist different levels of CO2/O2 specificity-determining factors, corresponding to different evolutionary events and specificity levels. Based on this assumption, we devised a scheme to identify these substrate-determining factors. From this analysis, we are able to discover different categories of the CO2/O2 specificity-determining factors that show which residue substitutions account for (relatively) small specificity changes, as happened in green plants, or a tremendous enhancement, as observed in marine non-green algae. Therefore, the analysis can improve our understanding of molecular mechanisms in the substrate specificity development and prioritize candidate specificity-determining surface residues for site-directed mutagenesis.

In silico discovery of enzyme-substrate specificity-determining residue clusters.

The binding between an enzyme and its substrate is highly specific, despite the fact that many different enzymes show significant sequence and structure similarity. There must be, then, substrate specificity-determining residues that enable different enzymes to recognize their unique substrates. We reason that a coordinated, not independent, action of both conserved and non-conserved residues determine enzymatic activity and specificity. Here, we present a surface patch ranking (SPR) method for in silico discovery of substrate specificity-determining residue clusters by exploring both sequence conservation and correlated mutations. As case studies we apply SPR to several highly homologous enzymatic protein pairs, such as guanylyl versus adenylyl cyclases, lactate versus malate dehydrogenases, and trypsin versus chymotrypsin. Without using experimental data, we predict several single and multi-residue clusters that are consistent with previous mutagenesis experimental results. Most single-residue clusters are directly involved in enzyme-substrate interactions, whereas multi-residue clusters are vital for domain-domain and regulator-enzyme interactions, indicating their complementary role in specificity determination. These results demonstrate that SPR may help the selection of target residues for mutagenesis experiments and, thus, focus rational drug design, protein engineering, and functional annotation to the relevant regions of a protein.

Protein dynamics and enzymatic catalysis: investigating the peptidyl-prolyl cis-trans isomerization activity of cyclophilin A.

A growing body of evidence suggests a connection between protein dynamics and enzymatic catalysis. In this paper, we present a variety of computational studies designed to investigate the role of protein dynamics in the detailed mechanism of peptidyl-prolyl cis-trans isomerization catalyzed by human cyclophilin A. The results identify a network of protein vibrations, extending from surface regions of the enzyme to the active site and coupled to substrate turnover. Indications are that this network may have a role in promoting catalysis. Crucial parts of this network are found to be conserved in 10 cyclophilin structures from six different species. Experimental evidence for the existence of this network comes from previous NMR relaxation studies, where motions in several residues, forming parts of this network, were detected only during substrate turnover. The high temperature factors (from X-ray crystal structures) associated with the network residues provide further evidence of these vibrations. Along with the knowledge of enzyme structure, this type of network could provide new insights into enzymatic catalysis and the effect of distant ligand binding on protein function. The procedure outlined in this paper is general and can be applied to other enzymatic systems as well. This presents an interesting opportunity; collaborative experimental and theoretical investigations designed to characterize in detail the nature and function of this type of network could enhance the understanding of protein dynamics in enzymatic catalysis.

An SVM-based algorithm for identification of photosynthesis-specific genome features.

This paper presents a novel algorithm for identification and functional characterization of "key" genome features responsible for a particular biochemical process of interest. The central idea is that individual genome features are identified as "key" features if the discrimination accuracy between two classes of genomes with respect to a given biochemical process is sufficiently affected by the inclusion or exclusion of these features. In this paper, genome features are defined by high-resolution gene functions. The discrimination procedure utilizes the Support Vector Machine classification technique. The application to the oxygenic photosynthetic process resulted in 126 highly confident candidate genome features. While many of these features are well-known components in the oxygenic photosynthetic process, others are completely unknown, even including some hypothetical proteins. It is obvious that our algorithm is capable of discovering features related to a targeted biochemical process.

Carbon sequestration in Synechococcus Sp.: from molecular machines to hierarchical modeling.

The U.S. Department of Energy recently announced the first five grants for the Genomes to Life (GTL) Program. The goal of this program is to "achieve the most far-reaching of all biological goals: a fundamental, comprehensive, and systematic understanding of life." While more information about the program can be found at the GTL website (www.doegenomestolife.org), this paper provides an overview of one of the five GTL projects funded, "Carbon Sequestration in Synechococcus Sp.: From Molecular Machines to Hierarchical Modeling." This project is a combined experimental and computational effort emphasizing developing, prototyping, and applying new computational tools and methods to elucidate the biochemical mechanisms of the carbon sequestration of Synechococcus Sp., an abundant marine cyanobacteria known to play an important role in the global carbon cycle. Understanding, predicting, and perhaps manipulating carbon fixation in the oceans has long been a major focus of biological oceanography and has more recently been of interest to a broader audience of scientists and policy makers. It is clear that the oceanic sinks and sources of CO(2) are important terms in the global environmental response to anthropogenic atmospheric inputs of CO(2) and that oceanic microorganisms play a key role in this response. However, the relationship between this global phenomenon and the biochemical mechanisms of carbon fixation in these microorganisms is poorly understood. The project includes five subprojects: an experimental investigation, three computational biology efforts, and a fifth which deals with addressing computational infrastructure challenges of relevance to this project and the Genomes to Life program as a whole. Our experimental effort is designed to provide biology and data to drive the computational efforts and includes significant investment in developing new experimental methods for uncovering protein partners, characterizing protein complexes, identifying new binding domains. We will also develop and apply new data measurement and statistical methods for analyzing microarray experiments. Our computational efforts include coupling molecular simulation methods with knowledge discovery from diverse biological data sets for high-throughput discovery and characterization of protein-protein complexes and developing a set of novel capabilities for inference of regulatory pathways in microbial genomes across multiple sources of information through the integration of computational and experimental technologies. These capabilities will be applied to Synechococcus regulatory pathways to characterize their interaction map and identify component proteins in these pathways. We will also investigate methods for combining experimental and computational results with visualization and natural language tools to accelerate discovery of regulatory pathways. Furthermore, given that the ultimate goal of this effort is to develop a systems-level of understanding of how the Synechococcus genome affects carbon fixation at the global scale, we will develop and apply a set of tools for capturing the carbon fixation behavior of complex of Synechococcus at different levels of resolution. Finally, because the explosion of data being produced by high-throughput experiments requires data analysis and models which are more computationally complex, more heterogeneous, and require coupling to ever increasing amounts of experimentally obtained data in varying formats, we have also established a companion computational infrastructure to support this effort as well as the Genomes to Life program as a whole.