SBMLToolkit.jl: a Julia package for importing SBML into the SciML ecosystem.

Julia is a general purpose programming language that was designed for simplifying and accelerating numerical analysis and computational science. In particular the Scientific Machine Learning (SciML) ecosystem of Julia packages includes frameworks for high-performance symbolic-numeric computations. It allows users to automatically enhance high-level descriptions of their models with symbolic preprocessing and automatic sparsification and parallelization of computations. This enables performant solution of differential equations, efficient parameter estimation and methodologies for automated model discovery with neural differential equations and sparse identification of nonlinear dynamics. To give the systems biology community easy access to SciML, we developed SBMLToolkit.jl. SBMLToolkit.jl imports dynamic SBML models into the SciML ecosystem to accelerate model simulation and fitting of kinetic parameters. By providing computational systems biologists with easy access to the open-source Julia ecosystevnm, we hope to catalyze the development of further Julia tools in this domain and the growth of the Julia bioscience community. SBMLToolkit.jl is freely available under the MIT license. The source code is available at https://github.com/SciML/SBMLToolkit.jl.

Forecasting virus outbreaks with social media data via neural ordinary differential equations.

During the Covid-19 pandemic, real-time social media data could in principle be used as an early predictor of a new epidemic wave. This possibility is examined here by employing a neural ordinary differential equation (neural ODE) trained to forecast viral outbreaks in a specific geographic region. It learns from multivariate time series of signals derived from a novel set of large online polls regarding COVID-19 symptoms. Once trained, the neural ODE can capture the dynamics of interconnected local signals and effectively estimate the number of new infections up to two months in advance. In addition, it may predict the future consequences of changes in the number of infected at a certain period, which might be related with the flow of individuals entering or exiting a region. This study provides persuasive evidence for the predictive ability of widely disseminated social media surveys for public health applications.

Julia for biologists.

Major computational challenges exist in relation to the collection, curation, processing and analysis of large genomic and imaging datasets, as well as the simulation of larger and more realistic models in systems biology. Here we discuss how a relative newcomer among programming languages-Julia-is poised to meet the current and emerging demands in the computational biosciences and beyond. Speed, flexibility, a thriving package ecosystem and readability are major factors that make high-performance computing and data analysis available to an unprecedented degree. We highlight how Julia's design is already enabling new ways of analyzing biological data and systems, and we provide a list of resources that can facilitate the transition into Julian computing.

Two heads are better than one: current landscape of integrating QSP and machine learning : An ISoP QSP SIG white paper by the working group on the integration of quantitative systems pharmacology and machine learning.

Quantitative systems pharmacology (QSP) modeling is applied to address essential questions in drug development, such as the mechanism of action of a therapeutic agent and the progression of disease. Meanwhile, machine learning (ML) approaches also contribute to answering these questions via the analysis of multi-layer 'omics' data such as gene expression, proteomics, metabolomics, and high-throughput imaging. Furthermore, ML approaches can also be applied to aspects of QSP modeling. Both approaches are powerful tools and there is considerable interest in integrating QSP modeling and ML. So far, a few successful implementations have been carried out from which we have learned about how each approach can overcome unique limitations of the other. The QSP + ML working group of the International Society of Pharmacometrics QSP Special Interest Group was convened in September, 2019 to identify and begin realizing new opportunities in QSP and ML integration. The working group, which comprises 21 members representing 18 academic and industry organizations, has identified four categories of current research activity which will be described herein together with case studies of applications to drug development decision making. The working group also concluded that the integration of QSP and ML is still in its early stages of moving from evaluating available technical tools to building case studies. This paper reports on this fast-moving field and serves as a foundation for future codification of best practices.

Stiff neural ordinary differential equations.

Neural Ordinary Differential Equations (ODEs) are a promising approach to learn dynamical models from time-series data in science and engineering applications. This work aims at learning neural ODEs for stiff systems, which are usually raised from chemical kinetic modeling in chemical and biological systems. We first show the challenges of learning neural ODEs in the classical stiff ODE systems of Robertson's problem and propose techniques to mitigate the challenges associated with scale separations in stiff systems. We then present successful demonstrations in stiff systems of Robertson's problem and an air pollution problem. The demonstrations show that the usage of deep networks with rectified activations, proper scaling of the network outputs as well as loss functions, and stabilized gradient calculations are the key techniques enabling the learning of stiff neural ODEs. The success of learning stiff neural ODEs opens up possibilities of using neural ODEs in applications with widely varying time-scales, such as chemical dynamics in energy conversion, environmental engineering, and life sciences.

Collocation based training of neural ordinary differential equations.

The predictive power of machine learning models often exceeds that of mechanistic modeling approaches. However, the interpretability of purely data-driven models, without any mechanistic basis is often complicated, and predictive power by itself can be a poor metric by which we might want to judge different methods. In this work, we focus on the relatively new modeling techniques of neural ordinary differential equations. We discuss how they relate to machine learning and mechanistic models, with the potential to narrow the gulf between these two frameworks: they constitute a class of hybrid model that integrates ideas from data-driven and dynamical systems approaches. Training neural ODEs as representations of dynamical systems data has its own specific demands, and we here propose a collocation scheme as a fast and efficient training strategy. This alleviates the need for costly ODE solvers. We illustrate the advantages that collocation approaches offer, as well as their robustness to qualitative features of a dynamical system, and the quantity and quality of observational data. We focus on systems that exemplify some of the hallmarks of complex dynamical systems encountered in systems biology, and we map out how these methods can be used in the analysis of mathematical models of cellular and physiological processes.

Safe Blues: The case for virtual safe virus spread in the long-term fight against epidemics.

Viral spread is a complicated function of biological properties, the environment, preventative measures such as sanitation and masks, and the rate at which individuals come within physical proximity. It is these last two elements that governments can control through social-distancing directives. However, infection measurements are almost always delayed, making real-time estimation nearly impossible. Safe Blues is one way of addressing the problem caused by this time lag via online measurements combined with machine learning methods that exploit the relationship between counts of multiple forms of the Safe Blues strands and the progress of the actual epidemic. The Safe Blues protocols and techniques have been developed together with an experimental minimal viable product, presented as an app on Android devices with a server backend. Following initial exploration via simulation experiments, we are now preparing for a university-wide experiment of Safe Blues.

Beyond Deterministic Models in Drug Discovery and Development.

The model-informed drug discovery and development paradigm is now well established among the pharmaceutical industry and regulatory agencies. This success has been mainly due to the ability of pharmacometrics to bring together different modeling strategies, such as population pharmacokinetics/pharmacodynamics (PK/PD) and systems biology/pharmacology. However, there are promising quantitative approaches that are still seldom used by pharmacometricians and that deserve consideration. One such case is the stochastic modeling approach, which can be important when modeling small populations because random events can have a huge impact on these systems. In this review, we aim to raise awareness of stochastic models and how to combine them with existing modeling techniques, with the ultimate goal of making future drug-disease models more versatile and realistic.

Mean-Independent Noise Control of Cell Fates via Intermediate States.

Stochasticity affects accurate signal detection and robust generation of correct cell fates. Although many known regulatory mechanisms may reduce fluctuations in signals, most simultaneously influence their mean dynamics, leading to unfaithful cell fates. Through analysis and computation, we demonstrate that a reversible signaling mechanism acting through intermediate states can reduce noise while maintaining the mean. This mean-independent noise control (MINC) mechanism is investigated in the context of an intracellular binding protein that regulates retinoic acid (RA) signaling during zebrafish hindbrain development. By comparing our models with experimental data, we find that the MINC mechanism allows for sharp boundaries of gene expression without sacrificing boundary accuracy. In addition, this MINC mechanism can modulate noise to levels that we show are beneficial to spatial patterning through noise-induced cell fate switching. These results reveal a design principle that may be important for noise regulation in many systems that control cell fate determination.

ADAPTIVE METHODS FOR STOCHASTIC DIFFERENTIAL EQUATIONS VIA NATURAL EMBEDDINGS AND REJECTION SAMPLING WITH MEMORY.

Adaptive time-stepping with high-order embedded Runge-Kutta pairs and rejection sampling provides efficient approaches for solving differential equations. While many such methods exist for solving deterministic systems, little progress has been made for stochastic variants. One challenge in developing adaptive methods for stochastic differential equations (SDEs) is the construction of embedded schemes with direct error estimates. We present a new class of embedded stochastic Runge-Kutta (SRK) methods with strong order 1.5 which have a natural embedding of strong order 1.0 methods. This allows for the derivation of an error estimate which requires no additional function evaluations. Next we derive a general method to reject the time steps without losing information about the future Brownian path termed Rejection Sampling with Memory (RSwM). This method utilizes a stack data structure to do rejection sampling, costing only a few floating point calculations. We show numerically that the methods generate statistically-correct and tolerance-controlled solutions. Lastly, we show that this form of adaptivity can be applied to systems of equations, and demonstrate that it solves a stiff biological model 12.28x faster than common fixed timestep algorithms. Our approach only requires the solution to a bridging problem and thus lends itself to natural generalizations beyond SDEs.

Noise modulation in retinoic acid signaling sharpens segmental boundaries of gene expression in the embryonic zebrafish hindbrain.

Morphogen gradients induce sharply defined domains of gene expression in a concentration-dependent manner, yet how cells interpret these signals in the face of spatial and temporal noise remains unclear. Using fluorescence lifetime imaging microscopy (FLIM) and phasor analysis to measure endogenous retinoic acid (RA) directly in vivo, we have investigated the amplitude of noise in RA signaling, and how modulation of this noise affects patterning of hindbrain segments (rhombomeres) in the zebrafish embryo. We demonstrate that RA forms a noisy gradient during critical stages of hindbrain patterning and that cells use distinct intracellular binding proteins to attenuate noise in RA levels. Increasing noise disrupts sharpening of rhombomere boundaries and proper patterning of the hindbrain. These findings reveal novel cellular mechanisms of noise regulation, which are likely to play important roles in other aspects of physiology and disease.