Reconstructing dynamical networks via feature ranking.

Empirical data on real complex systems are becoming increasingly available. Parallel to this is the need for new methods of reconstructing (inferring) the structure of networks from time-resolved observations of their node-dynamics. The methods based on physical insights often rely on strong assumptions about the properties and dynamics of the scrutinized network. Here, we use the insights from machine learning to design a new method of network reconstruction that essentially makes no such assumptions. Specifically, we interpret the available trajectories (data) as "features" and use two independent feature ranking approaches-Random Forest and RReliefF-to rank the importance of each node for predicting the value of each other node, which yields the reconstructed adjacency matrix. We show that our method is fairly robust to coupling strength, system size, trajectory length, and noise. We also find that the reconstruction quality strongly depends on the dynamical regime.

Equation discovery for systems biology: finding the structure and dynamics of biological networks from time course data.

Reconstructing biological networks, such as metabolic and signaling networks, is at the heart of systems biology. Although many approaches exist for reconstructing network structure, few approaches recover the full dynamic behavior of a network. We survey such approaches that originate from computational scientific discovery, a subfield of machine learning. These take as input measured time course data, as well as existing domain knowledge, such as partial knowledge of the network structure. We demonstrate the use of these approaches on illustrative tasks of finding the complete dynamics of biological networks, which include examples of rediscovering known networks and their dynamics, as well as examples of proposing models for unknown networks.

Extensive evaluation of the generalized relevance network approach to inferring gene regulatory networks.

Background: The generalized relevance network approach to network inference reconstructs network links based on the strength of associations between data in individual network nodes. It can reconstruct undirected networks, i.e., relevance networks, sensu stricto, as well as directed networks, referred to as causal relevance networks. The generalized approach allows the use of an arbitrary measure of pairwise association between nodes, an arbitrary scoring scheme that transforms the associations into weights of the network links, and a method for inferring the directions of the links. While this makes the approach powerful and flexible, it introduces the challenge of finding a combination of components that would perform well on a given inference task. Results: We address this challenge by performing an extensive empirical analysis of the performance of 114 variants of the generalized relevance network approach on 47 tasks of gene network inference from time-series data and 39 tasks of gene network inference from steady-state data. We compare the different variants in a multi-objective manner, considering their ranking in terms of different performance metrics. The results suggest a set of recommendations that provide guidance for selecting an appropriate variant of the approach in different data settings. Conclusions: The association measures based on correlation, combined with a particular scoring scheme of asymmetric weighting, lead to optimal performance of the relevance network approach in the general case. In the two special cases of inference tasks involving short time-series data and/or large networks, association measures based on identifying qualitative trends in the time series are more appropriate.

Classification tree analysis of second neoplasms in survivors of childhood cancer.

BACKGROUND: Reports on childhood cancer survivors estimated cumulative probability of developing secondary neoplasms vary from 3.3% to 25% at 25 years from diagnosis, and the risk of developing another cancer to several times greater than in the general population. METHODS: In our retrospective study, we have used the classification tree multivariate method on a group of 849 first cancer survivors, to identify childhood cancer patients with the greatest risk for development of secondary neoplasms. RESULTS: In observed group of patients, 34 develop secondary neoplasm after treatment of primary cancer. Analysis of parameters present at the treatment of first cancer, exposed two groups of patients at the special risk for secondary neoplasm. First are female patients treated for Hodgkin's disease at the age between 10 and 15 years, whose treatment included radiotherapy. Second group at special risk were male patients with acute lymphoblastic leukemia who were treated at the age between 4.6 and 6.6 years of age. CONCLUSION: The risk groups identified in our study are similar to the results of studies that used more conventional approaches. Usefulness of our approach in study of occurrence of second neoplasms should be confirmed in larger sample study, but user friendly presentation of results makes it attractive for further studies.

Constructing explanatory process models from biological data and knowledge.

OBJECTIVE: We address the task of inducing explanatory models from observations and knowledge about candidate biological processes, using the illustrative problem of modeling photosynthesis regulation. METHODS: We cast both models and background knowledge in terms of processes that interact to account for behavior. We also describe IPM, an algorithm for inducing quantitative process models from such input. RESULTS: We demonstrate IPM's use both on photosynthesis and on a second domain, biochemical kinetics, reporting the models induced and their fit to observations. CONCLUSION: We consider the generality of our approach, discuss related research on biological modeling, and suggest directions for future work.

Learning stochastic process-based models of dynamical systems from knowledge and data.

BACKGROUND: Identifying a proper model structure, using methods that address both structural and parameter uncertainty, is a crucial problem within the systems approach to biology. And yet, it has a marginal presence in the recent literature. While many existing approaches integrate methods for simulation and parameter estimation of a single model to address parameter uncertainty, only few of them address structural uncertainty at the same time. The methods for handling structure uncertainty often oversimplify the problem by allowing the human modeler to explicitly enumerate a relatively small number of alternative model structures. On the other hand, process-based modeling methods provide flexible modular formalisms for specifying large classes of plausible model structures, but their scope is limited to deterministic models. Here, we aim at extending the scope of process-based modeling methods to inductively learn stochastic models from knowledge and data. RESULTS: We combine the flexibility of process-based modeling in terms of addressing structural uncertainty with the benefits of stochastic modeling. The proposed method combines search trough the space of plausible model structures, the parsimony principle and parameter estimation to identify a model with optimal structure and parameters. We illustrate the utility of the proposed method on four stochastic modeling tasks in two domains: gene regulatory networks and epidemiology. Within the first domain, using synthetically generated data, the method successfully recovers the structure and parameters of known regulatory networks from simulations. In the epidemiology domain, the method successfully reconstructs previously established models of epidemic outbreaks from real, sparse and noisy measurement data. CONCLUSIONS: The method represents a unified approach to modeling dynamical systems that allows for flexible formalization of the space of candidate model structures, deterministic and stochastic interpretation of model dynamics, and automated induction of model structure and parameters from data. The method is able to reconstruct models of dynamical systems from synthetic and real data.

Modeling Dynamic Systems with Efficient Ensembles of Process-Based Models.

Ensembles are a well established machine learning paradigm, leading to accurate and robust models, predominantly applied to predictive modeling tasks. Ensemble models comprise a finite set of diverse predictive models whose combined output is expected to yield an improved predictive performance as compared to an individual model. In this paper, we propose a new method for learning ensembles of process-based models of dynamic systems. The process-based modeling paradigm employs domain-specific knowledge to automatically learn models of dynamic systems from time-series observational data. Previous work has shown that ensembles based on sampling observational data (i.e., bagging and boosting), significantly improve predictive performance of process-based models. However, this improvement comes at the cost of a substantial increase of the computational time needed for learning. To address this problem, the paper proposes a method that aims at efficiently learning ensembles of process-based models, while maintaining their accurate long-term predictive performance. This is achieved by constructing ensembles with sampling domain-specific knowledge instead of sampling data. We apply the proposed method to and evaluate its performance on a set of problems of automated predictive modeling in three lake ecosystems using a library of process-based knowledge for modeling population dynamics. The experimental results identify the optimal design decisions regarding the learning algorithm. The results also show that the proposed ensembles yield significantly more accurate predictions of population dynamics as compared to individual process-based models. Finally, while their predictive performance is comparable to the one of ensembles obtained with the state-of-the-art methods of bagging and boosting, they are substantially more efficient.

Process-based design of dynamical biological systems.

The computational design of dynamical systems is an important emerging task in synthetic biology. Given desired properties of the behaviour of a dynamical system, the task of design is to build an in-silico model of a system whose simulated be- haviour meets these properties. We introduce a new, process-based, design methodology for addressing this task. The new methodology combines a flexible process-based formalism for specifying the space of candidate designs with multi-objective optimization approaches for selecting the most appropriate among these candidates. We demonstrate that the methodology is general enough to both formulate and solve tasks of designing deterministic and stochastic systems, successfully reproducing plausible designs reported in previous studies and proposing new designs that meet the design criteria, but have not been previously considered.

Domain-specific model selection for structural identification of the Rab5-Rab7 dynamics in endocytosis.

BACKGROUND: Given its recent rapid development and the central role that modeling plays in the discipline, systems biology clearly needs methods for automated modeling of dynamical systems. Process-based modeling focuses on explanatory models of dynamical systems; it constructs such models from measured time-course data and formalized modeling knowledge. In this paper, we apply process-based modeling to the practically relevant task of modeling the Rab5-Rab7 conversion switch in endocytosis. The task is difficult due to the limited observability of the system variables and the noisy measurements, which pose serious challenges to the process of model selection. To address these issues, we propose a domain-specific model selection criteria that take into account knowledge about the necessary properties of the simulated model behavior. RESULTS: In a series of modeling experiments, we compare the results of process-based modeling obtained with different model selection criteria. The first is the standard maximum likelihood criterion based solely on least-squares model error. The second one is a parsimony-based criterion that also takes into account model complexity. We also introduce three domain-specific criteria based on domain expert expectations about the simulated behavior of an endocytosis model. According to the first criterion, 90 of the candidate models are indistinguishable. Furthermore, taking into account the complexity of the model does not lead to better model selection. However, the use of domain-specific criteria results in a remarkable improvement over the other two model selection criteria. CONCLUSIONS: We demonstrate the applicability of process-based modeling to the task of modeling the Rab5-Rab7 dynamics in endocytosis. Our experiments show that the domain-specific criteria outperform the standard domain-independent criteria for model selection. We also find that some of the model structures discarded as implausible in previous studies lead to the expected Rab5-Rab7 switch behavior.

Late somatic sequelae after treatment of childhood cancer in Slovenia.

BACKGROUND: This is a long-term follow-up clinical study of adolescents and adults, survivors of childhood cancer. We evaluate and analyze the late somatic sequelae of childhood cancer treatment. Many such studies are susceptible to a strong selection bias, i.e., they employ a limited non-systematic sample of patients, based on a clinical hospital that provided the cancer treatment or performed the follow-up. To address the issue of selection bias, we perform here an analysis of late sequelae on a systematic database of the entire population of the children treated for cancer in Slovenia. Due to the specifics of cancer treatment procedures in Slovenia, they have all been treated and followed-up in the same clinic. METHODS: The data are based on the centralized registry of cancer patients in Slovenia and present a controlled and homogeneous collection. Late sequelae are evaluated following a modified CTCAE, i.e., the National Cancer Institute's Common Terminology Criteria for Adverse Events version 3.0. We use survival analysis method to estimate the incidence of and risk for late sequelae, where the time variable is measured in years from the diagnosis date, while we follow the event of incidence of late sequelae scored other than none. Survival analysis is performed using Kaplan Meier estimator and Cox regression model. RESULTS: The incidence of mild, moderate, or severe late sequelae of childhood cancer treatment significantly decreased from 75% in the group of patients diagnosed before 1975 to 55% for those diagnosed after 1995. The Cox regression analysis of the risk factors for the incidence of late sequelae identifies three significant factors: treatment modalities, age at diagnosis, and primary diagnosis. CONCLUSIONS: The change of treatment modalities in terms of replacement of surgery and radiotherapy with chemotherapy is the main reason for the decrease of the incidence and the risk for late sequelae of childhood cancer treatment; treatment modalities including surgery significantly increase the risk ratio of late sequelae, while those based on chemotherapy only significantly decreases the risk. Risk of late sequelae increases with the diagnosis age: younger children are more susceptible to late effects of treatment. Finally, primary diagnosis significantly influences the risk for late sequelae, but mostly due to the dependency of the treatment modality on the primary diagnosis.

Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis.

BACKGROUND: We address the task of parameter estimation in models of the dynamics of biological systems based on ordinary differential equations (ODEs) from measured data, where the models are typically non-linear and have many parameters, the measurements are imperfect due to noise, and the studied system can often be only partially observed. A representative task is to estimate the parameters in a model of the dynamics of endocytosis, i.e., endosome maturation, reflected in a cut-out switch transition between the Rab5 and Rab7 domain protein concentrations, from experimental measurements of these concentrations. The general parameter estimation task and the specific instance considered here are challenging optimization problems, calling for the use of advanced meta-heuristic optimization methods, such as evolutionary or swarm-based methods. RESULTS: We apply three global-search meta-heuristic algorithms for numerical optimization, i.e., differential ant-stigmergy algorithm (DASA), particle-swarm optimization (PSO), and differential evolution (DE), as well as a local-search derivative-based algorithm 717 (A717) to the task of estimating parameters in ODEs. We evaluate their performance on the considered representative task along a number of metrics, including the quality of reconstructing the system output and the complete dynamics, as well as the speed of convergence, both on real-experimental data and on artificial pseudo-experimental data with varying amounts of noise. We compare the four optimization methods under a range of observation scenarios, where data of different completeness and accuracy of interpretation are given as input. CONCLUSIONS: Overall, the global meta-heuristic methods (DASA, PSO, and DE) clearly and significantly outperform the local derivative-based method (A717). Among the three meta-heuristics, differential evolution (DE) performs best in terms of the objective function, i.e., reconstructing the output, and in terms of convergence. These results hold for both real and artificial data, for all observability scenarios considered, and for all amounts of noise added to the artificial data. In sum, the meta-heuristic methods considered are suitable for estimating the parameters in the ODE model of the dynamics of endocytosis under a range of conditions: With the model and conditions being representative of parameter estimation tasks in ODE models of biochemical systems, our results clearly highlight the promise of bio-inspired meta-heuristic methods for parameter estimation in dynamic system models within system biology.

Users' information-seeking behavior on a medical library Website.

The Central Medical Library (CMK) at the Faculty of Medicine, University of Ljubljana, Slovenia, started to build a library Website that included a guide to library services and resources in 1997. The evaluation of Website usage plays an important role in its maintenance and development. Analyzing and exploring regularities in the visitors' behavior can be used to enhance the quality and facilitate delivery of information services, identify visitors' interests, and improve the server's performance. The analysis of the CMK Website users' navigational behavior was carried out by analyzing the Web server log files. These files contained information on all user accesses to the Website and provided a great opportunity to learn more about the behavior of visitors to the Website. The majority of the available tools for Web log file analysis provide a predefined set of reports showing the access count and the transferred bytes grouped along several dimensions. In addition to the reports mentioned above, the authors wanted to be able to perform interactive exploration and ad hoc analysis and discover trends in a user-friendly way. Because of that, we developed our own solution for exploring and analyzing the Web logs based on data warehousing and online analytical processing technologies. The analytical solution we developed proved successful, so it may find further application in the field of Web log file analysis. We will apply the findings of the analysis to restructuring the CMK Website.