Deep reinforcement learning for optimal experimental design in biology.

The field of optimal experimental design uses mathematical techniques to determine experiments that are maximally informative from a given experimental setup. Here we apply a technique from artificial intelligence-reinforcement learning-to the optimal experimental design task of maximizing confidence in estimates of model parameter values. We show that a reinforcement learning approach performs favourably in comparison with a one-step ahead optimisation algorithm and a model predictive controller for the inference of bacterial growth parameters in a simulated chemostat. Further, we demonstrate the ability of reinforcement learning to train over a distribution of parameters, indicating that this approach is robust to parametric uncertainty.

Calibrating spatiotemporal models of microbial communities to microscopy data: A review.

Spatiotemporal models that account for heterogeneity within microbial communities rely on single-cell data for calibration and validation. Such data, commonly collected via microscopy and flow cytometry, have been made more accessible by recent advances in microfluidics platforms and data processing pipelines. However, validating models against such data poses significant challenges. Validation practices vary widely between modelling studies; systematic and rigorous methods have not been widely adopted. Similar challenges are faced by the (macrobial) ecology community, in which systematic calibration approaches are often employed to improve quantitative predictions from computational models. Here, we review single-cell observation techniques that are being applied to study microbial communities and the calibration strategies that are being employed for accompanying spatiotemporal models. To facilitate future calibration efforts, we have compiled a list of summary statistics relevant for quantifying spatiotemporal patterns in microbial communities. Finally, we highlight some recently developed techniques that hold promise for improved model calibration, including algorithmic guidance of summary statistic selection and machine learning approaches for efficient model simulation.

A Keller-Segel model for C elegans L1 aggregation.

We describe a mathematical model for the aggregation of starved first-stage C elegans larvae (L1s). We propose that starved L1s produce and respond chemotactically to two labile diffusible chemical signals, a short-range attractant and a longer range repellent. This model takes the mathematical form of three coupled partial differential equations, one that describes the movement of the worms and one for each of the chemical signals. Numerical solution of these equations produced a pattern of aggregates that resembled that of worm aggregates observed in experiments. We also describe the identification of a sensory receptor gene, srh-2, whose expression is induced under conditions that promote L1 aggregation. Worms whose srh-2 gene has been knocked out form irregularly shaped aggregates. Our model suggests this phenotype may be explained by the mutant worms slowing their movement more quickly than the wild type.

Deep reinforcement learning for the control of microbial co-cultures in bioreactors.

Multi-species microbial communities are widespread in natural ecosystems. When employed for biomanufacturing, engineered synthetic communities have shown increased productivity in comparison with monocultures and allow for the reduction of metabolic load by compartmentalising bioprocesses between multiple sub-populations. Despite these benefits, co-cultures are rarely used in practice because control over the constituent species of an assembled community has proven challenging. Here we demonstrate, in silico, the efficacy of an approach from artificial intelligence-reinforcement learning-for the control of co-cultures within continuous bioreactors. We confirm that feedback via a trained reinforcement learning agent can be used to maintain populations at target levels, and that model-free performance with bang-bang control can outperform a traditional proportional integral controller with continuous control, when faced with infrequent sampling. Further, we demonstrate that a satisfactory control policy can be learned in one twenty-four hour experiment by running five bioreactors in parallel. Finally, we show that reinforcement learning can directly optimise the output of a co-culture bioprocess. Overall, reinforcement learning is a promising technique for the control of microbial communities.

Advancing undergraduate synthetic biology education: insights from a Canadian iGEM student perspective.

The last two decades have seen vigorous activity in synthetic biology research and the ever-increasing applications of these technologies. However, pedagogical research pertaining to teaching synthetic biology is scarce, especially when compared to other science and engineering disciplines. Within Canada, there are only three universities that offer synthetic biology programs, two of which are at the undergraduate level. Rather than taking place in formal academic settings, many Canadian undergraduate students are introduced to synthetic biology through participation in the annual International Genetically Engineered Machine (iGEM) competition. Although the iGEM competition has had a transformative impact on synthetic biology training in other nations, its impact in Canada has been relatively modest. Consequently, the iGEM competition remains a major setting for synthetic biology education in Canada. To promote further development of synthetic biology education, we surveyed undergraduate students from the Canadian iGEM design teams of 2019. We extracted insights from these data using qualitative analysis to provide recommendations for best teaching practices in synthetic biology undergraduate education, which we describe through our proposed Framework for Transdisciplinary Synthetic Biology Education (FTSBE).

Gene-Centric Model Approaches for Accurate Prediction of Pesticide Biodegradation in Soils.

Pesticides are widely used in agriculture despite their negative impact on ecosystems and human health. Biogeochemical modeling facilitates the mechanistic understanding of microbial controls on pesticide turnover in soils. We propose to inform models of coupled microbial dynamics and pesticide turnover with measurements of the abundance and expression of functional genes. To assess the advantages of informing models with genetic data, we developed a novel "gene-centric" model and compared model variants of differing structural complexity against a standard biomass-based model. The models were calibrated and validated using data from two batch experiments in which the degradation of the pesticides dichlorophenoxyacetic acid (2,4-D) and 2-methyl-4-chlorophenoxyacetic acid (MCPA) were observed in soil. When calibrating against data on pesticide mineralization, the gene-centric and biomass-based models performed equally well. However, accounting for pesticide-triggered gene regulation allows improved performance in capturing microbial dynamics and in predicting pesticide mineralization. This novel modeling approach also reveals a hysteretic relationship between pesticide degradation rates and gene expression, implying that the biodegradation performance in soils cannot be directly assessed by measuring the expression of functional genes. Our gene-centric model provides an effective approach for exploiting molecular biology data to simulate pesticide degradation in soils.

Parametric Sensitivity Analysis of Oscillatory Delay Systems with an Application to Gene Regulation.

A parametric sensitivity analysis for periodic solutions of delay-differential equations is developed. Because phase shifts cause the sensitivity coefficients of a periodic orbit to diverge, we focus on sensitivities of the extrema, from which amplitude sensitivities are computed, and of the period. Delay-differential equations are often used to model gene expression networks. In these models, the parametric sensitivities of a particular genotype define the local geometry of the evolutionary landscape. Thus, sensitivities can be used to investigate directions of gradual evolutionary change. An oscillatory protein synthesis model whose properties are modulated by RNA interference is used as an example. This model consists of a set of coupled delay-differential equations involving three delays. Sensitivity analyses are carried out at several operating points. Comments on the evolutionary implications of the results are offered.

Microbial herd protection mediated by antagonistic interaction in polymicrobial communities.

In the host and natural environments, microbes often exist in complex multispecies communities. The molecular mechanisms through which such communities develop and persist - despite significant antagonistic interactions between species - are not well understood. The type VI secretion system (T6SS) is a lethal weapon commonly employed by Gram-negative bacteria to inhibit neighboring species through delivery of toxic effectors. It is well established that intra-species protection is conferred by immunity proteins that neutralize effector toxicities. By contrast, the mechanisms for interspecies protection are not clear. Here we use two T6SS active antagonistic bacteria, Aeromonas hydrophila (AH) and Vibrio cholerae (VC), to demonstrate that interspecies protection is dependent on effectors. AH and VC do not share conserved immunity genes but could equally co-exist in a mixture. However, mutants lacking the T6SS or effectors were effectively eliminated by the other competing wild type. Time-lapse microscopy analyses show that mutually lethal interactions drive the segregation of mixed species into distinct single-species clusters by eliminating interspersed single cells. Cluster formation provides herd protection by abolishing lethal interaction inside each cluster and restricting it to the boundary. Using an agent-based modeling approach, we simulated the antagonistic interactions of two hypothetical species. The resulting simulations recapitulate our experimental observation. These results provide mechanistic insights for the general role of microbial weapons in determining the structures of complex multispecies communities. IMPORTANCE: Investigating the warfare of microbes allows us to better understand the ecological relationships in complex microbial communities such as the human microbiota. Here we use the T6SS, a deadly bacterial weapon, as a model to demonstrate the importance of lethal interactions in determining community structures and exchange of genetic materials. This simplified model elucidates a mechanism of microbial herd protection by which competing antagonistic species coexist in the same niche despite their diverse mutually destructive activities. Our results also suggest that antagonistic interaction imposes a strong selection that could promote multicellular like social behaviors and contribute to the transition to multicellularity during evolution.

Exploiting stoichiometric redundancies for computational efficiency and network reduction.

Analysis of metabolic networks typically begins with construction of the stoichiometry matrix, which characterizes the network topology. This matrix provides, via the balance equation, a description of the potential steady-state flow distribution. This paper begins with the observation that the balance equation depends only on the structure of linear redundancies in the network, and so can be stated in a succinct manner, leading to computational efficiencies in steady-state analysis. This alternative description of steady-state behaviour is then used to provide a novel method for network reduction, which complements existing algorithms for describing intracellular networks in terms of input-output macro-reactions (to facilitate bioprocess optimization and control). Finally, it is demonstrated that this novel reduction method can be used to address elementary mode analysis of large networks: the modes supported by a reduced network can capture the input-output modes of a metabolic module with significantly reduced computational effort.

Heterologous expression of NoxA confers aerotolerance in Clostridium sporogenes.

Clostridium is a genus of gram-positive obligate anaerobic bacteria. Some species of Clostridium, including Clostridium sporogenes, may be of use in bacteria-mediated cancer therapy. Spores of Clostridium are inert in healthy normoxic tissue but germinate when in the hypoxic regions of solid tumors, causing tumor regression. However, such treatments fail to completely eradicate tumors partly because of higher oxygen levels at the tumor's outer rim. In this study, we demonstrate that a degree of aerotolerance can be introduced to C. sporogenes by transfer of the noxA gene from Clostridium aminovalericum. NoxA is a water-forming NADH oxidase enzyme, and so has no detrimental effect on cell viability. In addition to its potential in cancer treatment, the noxA-expressing strain described here could be used to alleviate challenges related to oxygen sensitivity of C. sporogenes in biomanufacturing.

Modeling the coupled extracellular and intracellular environments in mammalian cell culture.

The regulation of metabolism in mammalian cell culture is closely linked to the process of apoptosis-programmed cell death. Apoptosis negatively impacts culture viability, product yield, and quality. An improved understanding of the interaction between apoptosis and metabolism will give rise to better control over the culture process, and thus improvements in product yield. This study presents a mathematical model that describes both the metabolic fluxes involving the extracellular metabolites and the progression of apoptosis in terms of intracellular caspases, and thus highlights the interactions between these two processes. The model is trained and validated against experimental observations of Chinese Hamster Ovary cell culture producing monoclonal antibody. Importantly, the model describes the continued production of monoclonal antibody in post exponential phase by incorporating different rates of antibody production for separate sub-populations within the culture. A parameter estimability test was applied on the combined model to assess the confidence in parameter estimates.

Kinetic modelling of ß-cell metabolism reveals control points in the insulin-regulating pyruvate cycling pathways.

Insulin, a key hormone in the regulation of glucose homoeostasis, is secreted by pancreatic ß-cells in response to elevated glucose levels. Insulin is released in a biphasic manner in response to glucose metabolism in ß-cells. The first phase of insulin secretion is triggered by an increase in the ATP:ADP ratio; the second phase occurs in response to both a rise in ATP:ADP and other key metabolic signals, including a rise in the NADPH:NADP+ ratio. Experimental evidence indicates that pyruvate-cycling pathways play an important role in the elevation of the NADPH:NADP+ ratio in response to glucose. The authors developed a kinetic model for the tricarboxylic acid cycle and pyruvate cycling pathways. The authors successfully validated the model against experimental observations and performed a sensitivity analysis to identify key regulatory interactions in the system. The model predicts that the dicarboxylate carrier and the pyruvate transporter are the most important regulators of pyruvate cycling and NADPH production. In contrast, the analysis showed that variation in the pyruvate carboxylase flux was compensated by a response in the activity of mitochondrial isocitrate dehydrogenase (ICDm ) resulting in minimal effect on overall pyruvate cycling flux. The model predictions suggest starting points for further experimental investigation, as well as potential drug targets for the treatment of type 2 diabetes.

Population-based modeling of the progression of apoptosis in mammalian cell culture.

The production of biopharmaceuticals from mammalian cell culture is hindered by apoptosis, which is the primary cause of cell death in these cultures. As a tool for optimization of culture yield, this study presents a population-based model describing the progression of apoptosis in a monoclonal antibody (mAb)-producing Chinese hamster ovary (CHO) cell culture. Because mAb production does not cease when apoptosis begins, the model was designed to incorporate subpopulations at various stages in the progression of apoptosis. The model was validated against intracellular measurements of caspase activity as well as cell density, nutrient levels, and toxic metabolites. Since the specific details of apoptotic mechanisms have not been elucidated in this cell line, we employed a model comparison analysis that suggests the most plausible pathways of activation.

NLoed: A Python Package for Nonlinear Optimal Experimental Design in Systems Biology.

Modeling in systems and synthetic biology relies on accurate parameter estimates and predictions. Accurate model calibration relies, in turn, on data and on how well suited the available data are to a particular modeling task. Optimal experimental design (OED) techniques can be used to identify experiments and data collection procedures that will most efficiently contribute to a given modeling objective. However, implementation of OED is limited by currently available software tools that are not well suited for the diversity of nonlinear models and non-normal data commonly encountered in biological research. Moreover, existing OED tools do not make use of the state-of-the-art numerical tools, resulting in inefficient computation. Here, we present the NLoed software package and demonstrate its use with in vivo data from an optogenetic system in Escherichia coli. NLoed is an open-source Python library providing convenient access to OED methods, with particular emphasis on experimental design for systems biology research. NLoed supports a wide variety of nonlinear, multi-input/output, and dynamic models and facilitates modeling and design of experiments over a wide variety of data types. To support OED investigations, the NLoed package implements maximum likelihood fitting and diagnostic tools, providing a comprehensive modeling workflow. NLoed offers an accessible, modular, and flexible OED tool set suited to the wide variety of experimental scenarios encountered in systems biology research. We demonstrate NLoed's capabilities by applying it to experimental design for characterization of a bacterial optogenetic system.

Robust L-type calcium current expression following heterozygous knockout of the Cav1.2 gene in adult mouse heart.

Mechanisms that contribute to maintaining expression of functional ion channels at relatively constant levels following perturbations of channel biosynthesis are likely to contribute significantly to the stability of electrophysiological systems in some pathological conditions. In order to examine the robustness of L-type calcium current expression, the response to changes in Ca²âº channel Cav1.2 gene dosage was studied in adult mice. Using a cardiac-specific inducible Cre recombinase system, Cav1.2 mRNA was reduced to 11 ± 1% of control values in homozygous floxed mice and the mice died rapidly (11.9 ± 3 days) after induction of gene deletion. In these homozygous knockout mice, echocardiographic analysis showed that myocardial contractility was reduced to 14 ± 1% of control values shortly before death. For these mice, no effective compensatory changes in ion channel gene expression were triggered following deletion of both Cav1.2 alleles, despite the dramatic decay in cardiac function. In contrast to the homozygote knockout mice, following knockout of only one Cav1.2 allele, cardiac function remained unchanged, as did survival.Cav1.2mRNAexpression in the left ventricle of heterozygous knockout mice was reduced to 58 ± 3% of control values and there was a 21 ± 2% reduction in Cav1.2 protein expression. There was no significant reduction in L-type Ca²âº current density in these mice. The results are consistent with a model of L-type calcium channel biosynthesis in which there are one or more saturated steps, which act to buffer changes in both total Cav1.2 protein and L-type current expression.

Differential chromatin proteomics of the MMS-induced DNA damage response in yeast.

BACKGROUND: Protein enrichment by sub-cellular fractionation was combined with differential-in-gel-electrophoresis (DIGE) to address the detection of the low abundance chromatin proteins in the budding yeast proteome. Comparisons of whole-cell extracts and chromatin fractions were used to provide a measure of the degree of chromatin association for individual proteins, which could be compared across sample treatments. The method was applied to analyze the effect of the DNA damaging agent methyl methanesulfonate (MMS) on levels of chromatin-associated proteins. RESULTS: Up-regulation of several previously characterized DNA damage checkpoint-regulated proteins, such as Rnr4, Rpa1 and Rpa2, was observed. In addition, several novel DNA damage responsive proteins were identified and assessed for genotoxic sensitivity using either DAmP (decreased abundance by mRNA perturbation) or knockout strains, including Acf2, Arp3, Bmh1, Hsp31, Lsp1, Pst2, Rnr4, Rpa1, Rpa2, Ste4, Ycp4 and Yrb1. A strain in which the expression of the Ran-GTPase binding protein Yrb1 was reduced was found to be hypersensitive to genotoxic stress. CONCLUSION: The described method was effective at unveiling chromatin-associated proteins that are less likely to be detected in the absence of fractionation. Several novel proteins with altered chromatin abundance were identified including Yrb1, pointing to a role for this nuclear import associated protein in DNA damage response.

Estimating the stochastic bifurcation structure of cellular networks.

High throughput measurement of gene expression at single-cell resolution, combined with systematic perturbation of environmental or cellular variables, provides information that can be used to generate novel insight into the properties of gene regulatory networks by linking cellular responses to external parameters. In dynamical systems theory, this information is the subject of bifurcation analysis, which establishes how system-level behaviour changes as a function of parameter values within a given deterministic mathematical model. Since cellular networks are inherently noisy, we generalize the traditional bifurcation diagram of deterministic systems theory to stochastic dynamical systems. We demonstrate how statistical methods for density estimation, in particular, mixture density and conditional mixture density estimators, can be employed to establish empirical bifurcation diagrams describing the bistable genetic switch network controlling galactose utilization in yeast Saccharomyces cerevisiae. These approaches allow us to make novel qualitative and quantitative observations about the switching behavior of the galactose network, and provide a framework that might be useful to extract information needed for the development of quantitative network models.

Considerations for using integral feedback control to construct a perfectly adapting synthetic gene network.

It has long been known to control theorists and engineers that integral feedback control leads to, and is necessary for, "perfect" adaptation to step input perturbations in most systems. Consequently, implementation of this robust control strategy in a synthetic gene network is an attractive prospect. However, the nature of genetic regulatory networks (density-dependent kinetics and molecular signals that easily reach saturation) implies that the design and construction of such a device is not straightforward. In this study, we propose a generic two-promoter genetic regulatory network for the purpose of exhibiting perfect adaptation; our treatment highlights the challenges inherent in the implementation of a genetic integral controller. We also present a numerical case study for a specific realization of this two-promoter network, "constructed" using commonly available parts from the bacterium Escherichia coli. We illustrate the possibility of optimizing this network's transient response via analogy to a linear, free-damped harmonic oscillator. Finally, we discuss extensions of this two-promoter network to a proportional-integral controller and to a three-promoter network capable of perfect adaptation under conditions where first-order protein removal effects would otherwise disrupt the adaptation.

Sensitivity analysis: from model parameters to system behaviour.

Sensitivity analysis addresses the manner in which model behaviour depends on model parametrization. Global sensitivity analysis makes use of statistical tools to address system behaviour over a wide range of operating conditions, whereas local sensitivity analysis focuses attention on a specific set of nominal parameter values. This narrow focus allows a complete analytical treatment and straightforward interpretation in the local case. Sensitivity analysis is a valuable tool for model construction and interpretation, and can be applied in medicine and biotechnology to predict the effect of interventions.