Reusable rule-based cell cycle model explains compartment-resolved dynamics of 16 observables in RPE-1 cells.

The mammalian cell cycle is regulated by a well-studied but complex biochemical reaction system. Computational models provide a particularly systematic and systemic description of the mechanisms governing mammalian cell cycle control. By combining both state-of-the-art multiplexed experimental methods and powerful computational tools, this work aims at improving on these models along four dimensions: model structure, validation data, validation methodology and model reusability. We developed a comprehensive model structure of the full cell cycle that qualitatively explains the behaviour of human retinal pigment epithelial-1 cells. To estimate the model parameters, time courses of eight cell cycle regulators in two compartments were reconstructed from single cell snapshot measurements. After optimisation with a parallel global optimisation metaheuristic we obtained excellent agreements between simulations and measurements. The PEtab specification of the optimisation problem facilitates reuse of model, data and/or optimisation results. Future perturbation experiments will improve parameter identifiability and allow for testing model predictive power. Such a predictive model may aid in drug discovery for cell cycle-related disorders.

IDESS: a toolbox for identification and automated design of stochastic gene circuits.

MOTIVATION: One of the main causes hampering predictability during the model identification and automated design of gene circuits in synthetic biology is the effect of molecular noise. Stochasticity may significantly impact the dynamics and function of gene circuits, specially in bacteria and yeast due to low mRNA copy numbers. Standard stochastic simulation methods are too computationally costly in realistic scenarios to be applied to optimization-based design or parameter estimation. RESULTS: In this work, we present IDESS (Identification and automated Design of Stochastic gene circuitS), a software toolbox for optimization-based design and model identification of gene regulatory circuits in the stochastic regime. This software incorporates an efficient approximation of the Chemical Master Equation as well as a stochastic simulation algorithm-both with GPU and CPU implementations-combined with global optimization algorithms capable of solving Mixed Integer Nonlinear Programming problems. The toolbox efficiently addresses two types of problems relevant in systems and synthetic biology: the automated design of stochastic synthetic gene circuits, and the parameter estimation for model identification of stochastic gene regulatory networks. AVAILABILITY AND IMPLEMENTATION: IDESS runs under the MATLAB environment and it is available under GPLv3 license at https://doi.org/10.5281/zenodo.7788692.

Distilling identifiable and interpretable dynamic models from biological data.

Mechanistic dynamical models allow us to study the behavior of complex biological systems. They can provide an objective and quantitative understanding that would be difficult to achieve through other means. However, the systematic development of these models is a non-trivial exercise and an open problem in computational biology. Currently, many research efforts are focused on model discovery, i.e. automating the development of interpretable models from data. One of the main frameworks is sparse regression, where the sparse identification of nonlinear dynamics (SINDy) algorithm and its variants have enjoyed great success. SINDy-PI is an extension which allows the discovery of rational nonlinear terms, thus enabling the identification of kinetic functions common in biochemical networks, such as Michaelis-Menten. SINDy-PI also pays special attention to the recovery of parsimonious models (Occam's razor). Here we focus on biological models composed of sets of deterministic nonlinear ordinary differential equations. We present a methodology that, combined with SINDy-PI, allows the automatic discovery of structurally identifiable and observable models which are also mechanistically interpretable. The lack of structural identifiability and observability makes it impossible to uniquely infer parameter and state variables, which can compromise the usefulness of a model by distorting its mechanistic significance and hampering its ability to produce biological insights. We illustrate the performance of our method with six case studies. We find that, despite enforcing sparsity, SINDy-PI sometimes yields models that are unidentifiable. In these cases we show how our method transforms their equations in order to obtain a structurally identifiable and observable model which is also interpretable.

Automated Design of Synthetic Gene Circuits in the Presence of Molecular Noise.

Microorganisms (mainly bacteria and yeast) are frequently used as hosts for genetic constructs in synthetic biology applications. Molecular noise might have a significant effect on the dynamics of gene regulation in microbial cells, mainly attributed to the low copy numbers of mRNA species involved. However, the inclusion of molecular noise in the automated design of biocircuits is not a common practice due to the computational burden linked to the chemical master equation describing the dynamics of stochastic gene regulatory circuits. Here, we address the automated design of synthetic gene circuits under the effect of molecular noise combining a mixed integer nonlinear global optimization method with a partial integro-differential equation model describing the evolution of stochastic gene regulatory systems that approximates very efficiently the chemical master equation. We demonstrate the performance of the proposed methodology through a number of examples of relevance in synthetic biology, including different bimodal stochastic gene switches, robust stochastic oscillators, and circuits capable of achieving biochemical adaptation under noise.

Optimal evaluation of energy yield and driving force in microbial metabolic pathway variants.

This work presents a methodology to evaluate the bioenergetic feasibility of alternative metabolic pathways for a given microbial conversion, optimising their energy yield and driving forces as a function of the concentration of metabolic intermediates. The tool, based on thermodynamic principles and multi-objective optimisation, accounts for pathway variants in terms of different electron carriers, as well as energy conservation (proton translocating) reactions within the pathway. The method also accommodates other constraints, some of them non-linear, such as the balance of conserved moieties. The approach involves the transformation of the maximum energy yield problem into a multi-objective mixed-integer linear optimisation problem which is then subsequently solved using the epsilon-constraint method, highlighting the trade-off between yield and rate in metabolic reactions. The methodology is applied to analyse several pathway alternatives occurring during propionate oxidation in anaerobic fermentation processes, as well as to the reverse TCA cycle pathway occurring during autotrophic microbial CO2 fixation. The results obtained using the developed methodology match previously reported literature and bring about insights into the studied pathways.

Automated design of gene circuits with optimal mushroom-bifurcation behavior.

Recent advances in synthetic biology are enabling exciting technologies, including the next generation of biosensors, the rational design of cell memory, modulated synthetic cell differentiation, and generic multifunctional biocircuits. These novel applications require the design of gene circuits leading to sophisticated behaviors and functionalities. At the same time, designs need to be kept minimal to avoid compromising cell viability. Bifurcation theory addresses such challenges by associating circuit dynamical properties with molecular details of its design. Nevertheless, incorporating bifurcation analysis into automated design processes has not been accomplished yet. This work presents an optimization-based method for the automated design of synthetic gene circuits with specified bifurcation diagrams that employ minimal network topologies. Using this approach, we designed circuits exhibiting the mushroom bifurcation, distilled the most robust topologies, and explored its multifunctional behavior. We then outline potential applications in biosensors, memory devices, and synthetic cell differentiation.

Global Optimization Approach for Parameter Estimation in Stochastic Dynamic Models of Biosystems.

Mechanistic dynamic models have become an essential tool for understanding biomolecular networks and other biological systems. Biochemical stochasticity can be extremely important in some situations, e.g., at the single-cell level where there is a low copy number of the species involved. In these scenarios, deterministic models are not suitable to characterize the dynamics, so stochastic dynamic models should be considered. Here, we address the challenging problem of parameter estimation in stochastic dynamic models. Despite recent advances, this area is considerably less mature than its deterministic counterpart. We present a novel strategy based on two components: (i) global optimization via a hybrid stochastic-deterministic approach, and (ii) stochastic simulation techniques tailored to the sparsity of the available experimental data. Regarding the latter, for cases of dense population data we make use of a novel approach using a Partial Integro-Differential Equation (PIDE) model solved using a semilagrangian method. In order to further speed up the simulations, we also present efficient parallel implementations for multi-core CPUs and also for graphical processing units (GPUs). Importantly, whereas SDE and Fokker Planck approximations of the Chemical Master Equation (CME) apply when the reactant populations are sufficiently large, the PIDE approximation to the CME is valid for very low copy numbers, and therefore they enable us to tackle parameter estimation for systems with large intrinsic molecular noise, (highly stochastic regimes far from the thermodynamic limit). We test our strategy with four challenging problems: a Lotka-Volterra system, a polarization system in S. cerevisiae, a genetic toggle switch, and a genetic circadian oscillator. Our method could successfully solve these problems in very reasonable computation times (often a few minutes for the first two problems) using standard low-cost hardware, showing very significant speedups with respect to recent alternative methods. The code used to obtain the results reported here is available at https://doi.org/10.5281/zenodo.5195408.

Improving dynamic predictions with ensembles of observable models.

MOTIVATION: Dynamic mechanistic modelling in systems biology has been hampered by the complexity and variability associated with the underlying interactions, and by uncertain and sparse experimental measurements. Ensemble modelling, a concept initially developed in statistical mechanics, has been introduced in biological applications with the aim of mitigating those issues. Ensemble modelling uses a collection of different models compatible with the observed data to describe the phenomena of interest. However, since systems biology models often suffer from a lack of identifiability and observability, ensembles of models are particularly unreliable when predicting non-observable states. RESULTS: We present a strategy to assess and improve the reliability of a class of model ensembles. In particular, we consider kinetic models described using ordinary differential equations with a fixed structure. Our approach builds an ensemble with a selection of the parameter vectors found when performing parameter estimation with a global optimization metaheuristic. This technique enforces diversity during the sampling of parameter space and it can quantify the uncertainty in the predictions of state trajectories. We couple this strategy with structural identifiability and observability analysis, and when these tests detect possible prediction issues we obtain model reparameterizations that surmount them. The end result is an ensemble of models with the ability to predict the internal dynamics of a biological process. We demonstrate our approach with models of glucose regulation, cell division, circadian oscillations and the JAK-STAT signalling pathway. AVAILABILITY AND IMPLEMENTATION: The code that implements the methodology and reproduces the results is available at https://doi.org/10.5281/zenodo.6782638. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Assessment of Prediction Uncertainty Quantification Methods in Systems Biology.

Biological processes are often modelled using ordinary differential equations. The unknown parameters of these models are estimated by optimizing the fit of model simulation and experimental data. The resulting parameter estimates inevitably possess some degree of uncertainty. In practical applications it is important to quantify these parameter uncertainties as well as the resulting prediction uncertainty, which are uncertainties of potentially time-dependent model characteristics. Unfortunately, estimating prediction uncertainties accurately is nontrivial, due to the nonlinear dependence of model characteristics on parameters. While a number of numerical approaches have been proposed for this task, their strengths and weaknesses have not been systematically assessed yet. To fill this knowledge gap, we apply four state of the art methods for uncertainty quantification to four case studies of different computational complexities. This reveals the trade-offs between their applicability and their statistical interpretability. Our results provide guidelines for choosing the most appropriate technique for a given problem and applying it successfully.

OptDesign: Identifying Optimum Design Strategies in Strain Engineering for Biochemical Production.

Computational tools have been widely adopted for strain optimization in metabolic engineering, contributing to numerous success stories of producing industrially relevant biochemicals. However, most of these tools focus on single metabolic intervention strategies (either gene/reaction knockout or amplification alone) and rely on hypothetical optimality principles (e.g., maximization of growth) and precise gene expression (e.g., fold changes) for phenotype prediction. This paper introduces OptDesign, a new two-step strain design strategy. In the first step, OptDesign selects regulation candidates that have a noticeable flux difference between the wild type and production strains. In the second step, it computes optimal design strategies with limited manipulations (combining regulation and knockout), leading to high biochemical production. The usefulness and capabilities of OptDesign are demonstrated for the production of three biochemicals in Escherichia coli using the latest genome-scale metabolic model iML1515, showing highly consistent results with previous studies while suggesting new manipulations to boost strain performance. The source code is available at https://github.com/chang88ye/OptDesign.

A protocol for dynamic model calibration.

Ordinary differential equation models are nowadays widely used for the mechanistic description of biological processes and their temporal evolution. These models typically have many unknown and nonmeasurable parameters, which have to be determined by fitting the model to experimental data. In order to perform this task, known as parameter estimation or model calibration, the modeller faces challenges such as poor parameter identifiability, lack of sufficiently informative experimental data and the existence of local minima in the objective function landscape. These issues tend to worsen with larger model sizes, increasing the computational complexity and the number of unknown parameters. An incorrectly calibrated model is problematic because it may result in inaccurate predictions and misleading conclusions. For nonexpert users, there are a large number of potential pitfalls. Here, we provide a protocol that guides the user through all the steps involved in the calibration of dynamic models. We illustrate the methodology with two models and provide all the code required to reproduce the results and perform the same analysis on new models. Our protocol provides practitioners and researchers in biological modelling with a one-stop guide that is at the same time compact and sufficiently comprehensive to cover all aspects of the problem.

PEtab-Interoperable specification of parameter estimation problems in systems biology.

Reproducibility and reusability of the results of data-based modeling studies are essential. Yet, there has been-so far-no broadly supported format for the specification of parameter estimation problems in systems biology. Here, we introduce PEtab, a format which facilitates the specification of parameter estimation problems using Systems Biology Markup Language (SBML) models and a set of tab-separated value files describing the observation model and experimental data as well as parameters to be estimated. We already implemented PEtab support into eight well-established model simulation and parameter estimation toolboxes with hundreds of users in total. We provide a Python library for validation and modification of a PEtab problem and currently 20 example parameter estimation problems based on recent studies.

Automated Biocircuit Design with SYNBADm.

SYNBADm is a Matlab toolbox for the automated design of biocircuits using a model-based optimization approach. It enables the design of biocircuits with pre-defined functions starting from libraries of biological parts. SYNBADm makes use of mixed integer global optimization and allows both single and multi-objective design problems. Here we describe a basic protocol for the design of synthetic gene regulatory circuits. We illustrate step-by-step how to solve two different problems: (1) the (single objective) design of a synthetic oscillator and (2) the (multi-objective) design of a circuit with switch-like behavior upon induction, with a good compromise between performance and protein production cost.

Structural identifiability and observability of compartmental models of the COVID-19 pandemic.

The recent coronavirus disease (COVID-19) outbreak has dramatically increased the public awareness and appreciation of the utility of dynamic models. At the same time, the dissemination of contradictory model predictions has highlighted their limitations. If some parameters and/or state variables of a model cannot be determined from output measurements, its ability to yield correct insights - as well as the possibility of controlling the system - may be compromised. Epidemic dynamics are commonly analysed using compartmental models, and many variations of such models have been used for analysing and predicting the evolution of the COVID-19 pandemic. In this paper we survey the different models proposed in the literature, assembling a list of 36 model structures and assessing their ability to provide reliable information. We address the problem using the control theoretic concepts of structural identifiability and observability. Since some parameters can vary during the course of an epidemic, we consider both the constant and time-varying parameter assumptions. We analyse the structural identifiability and observability of all of the models, considering all plausible choices of outputs and time-varying parameters, which leads us to analyse 255 different model versions. We classify the models according to their structural identifiability and observability under the different assumptions and discuss the implications of the results. We also illustrate with an example several alternative ways of remedying the lack of observability of a model. Our analyses provide guidelines for choosing the most informative model for each purpose, taking into account the available knowledge and measurements.

Synthetic Gene Circuit Analysis and Optimization.

Synthetic biology aims at engineering synthetic circuits with pre-defined target functions. From a systems (model-based) perspective, the following problems are of central importance: (1) given the model of a biomolecular circuit, elucidate whether it is capable of a certain behavior/functionality; and (2) starting from a pre-defined required functionality and a library of biological parts, find the biomolecular circuit that, built as a combination of such parts, achieves the desired behavior. These two problems, framed, respectively, as nonlinear analysis and automated design problems, are tackled here by efficient optimization methods. We illustrate these methods with case studies considering the analysis and design of biocircuits capable of bistability (bistable switches). Bistability is of particular interest in the context of systems and synthetic biology because it endows cells with the capacity to make decisions.

Using optimal control to understand complex metabolic pathways.

BACKGROUND: Optimality principles have been used to explain the structure and behavior of living matter at different levels of organization, from basic phenomena at the molecular level, up to complex dynamics in whole populations. Most of these studies have assumed a single-criteria approach. Such optimality principles have been justified from an evolutionary perspective. In the context of the cell, previous studies have shown how dynamics of gene expression in small metabolic models can be explained assuming that cells have developed optimal adaptation strategies. Most of these works have considered rather simplified representations, such as small linear pathways, or reduced networks with a single branching point, and a single objective for the optimality criteria. RESULTS: Here we consider the extension of this approach to more realistic scenarios, i.e. biochemical pathways of arbitrary size and structure. We first show that exploiting optimality principles for these networks poses great challenges due to the complexity of the associated optimal control problems. Second, in order to surmount such challenges, we present a computational framework which has been designed with scalability and efficiency in mind, including mechanisms to avoid the most common pitfalls. Third, we illustrate its performance with several case studies considering the central carbon metabolism of S. cerevisiae and B. subtilis. In particular, we consider metabolic dynamics during nutrient shift experiments. CONCLUSIONS: We show how multi-objective optimal control can be used to predict temporal profiles of enzyme activation and metabolite concentrations in complex metabolic pathways. Further, we also show how to consider general cost/benefit trade-offs. In this study we have considered metabolic pathways, but this computational framework can also be applied to analyze the dynamics of other complex pathways, such as signal transduction or gene regulatory networks.

A dual-parameter identification approach for data-based predictive modeling of hybrid gene regulatory network-growth kinetics in Pseudomonas putida mt-2.

Data integration to model-based description of biological systems incorporating gene dynamics improves the performance of microbial systems. Bioprocess performance, typically predicted using empirical Monod-type models, is essential for a sustainable bioeconomy. To replace empirical models, we updated a hybrid gene regulatory network-growth kinetic model, predicting aromatic pollutants degradation and biomass growth in Pseudomonas putida mt-2. We modeled a complex biological system including extensive information to understand the role of the regulatory elements in toluene biodegradation and biomass growth. The updated model exhibited extra complications such as the existence of oscillations and discontinuities. As parameter estimation of complex biological models remains a key challenge, we used the updated model to present a dual-parameter identification approach (the 'dual approach') combining two independent methodologies. Approach I handled the complexity by incorporation of demonstrated biological knowledge in the model-development process and combination of global sensitivity analysis and optimisation. Approach II complemented Approach I handling multimodality, ill-conditioning and overfitting through regularisation estimation, global optimisation, and identifiability analysis. To systematically quantify the biological system, we used a vast amount of high-quality time-course data. The dual approach resulted in an accurately calibrated kinetic model (NRMSE: 0.17055) efficiently handling the additional model complexity. We tested model validation using three independent experimental data sets, achieving greater predictive power (NRMSE: 0.18776) than the individual approaches (NRMSE I: 0.25322, II: 0.25227) and increasing model robustness. These results demonstrated data-driven predictive modeling potentially leading to bioprocess' model-based control and optimisation.

Full observability and estimation of unknown inputs, states and parameters of nonlinear biological models.

In this paper, we address the system identification problem in the context of biological modelling. We present and demonstrate a methodology for (i) assessing the possibility of inferring the unknown quantities in a dynamic model and (ii) effectively estimating them from output data. We introduce the term Full Input-State-Parameter Observability (FISPO) analysis to refer to the simultaneous assessment of state, input and parameter observability (note that parameter observability is also known as identifiability). This type of analysis has often remained elusive in the presence of unmeasured inputs. The method proposed in this paper can be applied to a general class of nonlinear ordinary differential equations models. We apply this approach to three models from the recent literature. First, we determine whether it is theoretically possible to infer the states, parameters and inputs, taking only the model equations into account. When this analysis detects deficiencies, we reformulate the model to make it fully observable. Then we move to numerical scenarios and apply an optimization-based technique to estimate the states, parameters and inputs. The results demonstrate the feasibility of an integrated strategy for (i) analysing the theoretical possibility of determining the states, parameters and inputs to a system and (ii) solving the practical problem of actually estimating their values.

Benchmarking optimization methods for parameter estimation in large kinetic models.

MOTIVATION: Kinetic models contain unknown parameters that are estimated by optimizing the fit to experimental data. This task can be computationally challenging due to the presence of local optima and ill-conditioning. While a variety of optimization methods have been suggested to surmount these issues, it is difficult to choose the best one for a given problem a priori. A systematic comparison of parameter estimation methods for problems with tens to hundreds of optimization variables is currently missing, and smaller studies provided contradictory findings. RESULTS: We use a collection of benchmarks to evaluate the performance of two families of optimization methods: (i) multi-starts of deterministic local searches and (ii) stochastic global optimization metaheuristics; the latter may be combined with deterministic local searches, leading to hybrid methods. A fair comparison is ensured through a collaborative evaluation and a consideration of multiple performance metrics. We discuss possible evaluation criteria to assess the trade-off between computational efficiency and robustness. Our results show that, thanks to recent advances in the calculation of parametric sensitivities, a multi-start of gradient-based local methods is often a successful strategy, but a better performance can be obtained with a hybrid metaheuristic. The best performer combines a global scatter search metaheuristic with an interior point local method, provided with gradients estimated with adjoint-based sensitivities. We provide an implementation of this method to render it available to the scientific community. AVAILABILITY AND IMPLEMENTATION: The code to reproduce the results is provided as Supplementary Material and is available at Zenodo https://doi.org/10.5281/zenodo.1304034. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.