Active Learning for Estimating Reachable Sets for Systems With Unknown Dynamics.

This article presents a data-driven method for computing reachable sets where active learning (AL) is used to reduce the computational burden. Set-based methods used to estimate reachable sets typically do not scale well with the state-space dimension, or rely heavily on the existence of a model. If such a model is not available, it is simple to generate state trajectory data by numerically simulating black-box oracles of systems (whose dynamics are unknown) from sampled initial conditions. Using these data samples, the estimation of reachable sets can be posed as a classification problem, wherein AL can intelligently select samples that are most informative and least similar to previously labeled samples. By exploiting submodularity, the actively learned samples can be selected efficiently, with bounded suboptimality. Our proposed framework is illustrated by estimating the domains of attractions of model predictive controllers (MPCs) and reinforcement learners. We also consider a scenario where there are two oracles that differ with respect to evaluation costs and labeling accuracy. We propose a framework to reduce the dependency of the expensive oracle in labeling samples using disagreement-based AL (DBAL). The potential of the DBAL algorithm is demonstrated on a solver selection problem for real-time MPC.

Safe Approximate Dynamic Programming via Kernelized Lipschitz Estimation.

We develop a method for obtaining safe initial policies for reinforcement learning via approximate dynamic programming (ADP) techniques for uncertain systems evolving with discrete-time dynamics. We employ the kernelized Lipschitz estimation to learn multiplier matrices that are used in semidefinite programming frameworks for computing admissible initial control policies with provably high probability. Such admissible controllers enable safe initialization and constraint enforcement while providing exponential stability of the equilibrium of the closed-loop system.

Embedded Model Predictive Control for a Wearable Artificial Pancreas.

While artificial pancreas (AP) systems are expected to improve the quality of life among people with type 1 diabetes mellitus (T1DM), the design of convenient systems that optimize the user experience, especially for those with active lifestyles, such as children and adolescents, still remains an open research question. In this work, we introduce an embeddable design and implementation of model predictive control (MPC) of AP systems for people with T1DM that significantly reduces the weight and on-body footprint of the AP system. The embeddable controller is based on a zone MPC that has been evaluated in multiple clinical studies. The proposed embedded zone MPC features a simpler design of the periodic safe zone in the cost function and the utilization of state-of-the-art alternating minimization algorithms for solving the convex programming problems inherent to MPC with linear models subject to convex constraints. Off-line closed-loop data generated by the FDA-accepted UVA/Padova simulator is used to select an optimization algorithm and corresponding tuning parameters. Through hardware-in-the-loop in silico results on a limited-resource Arduino Zero (Feather M0) platform, we demonstrate the potential of the proposed embedded MPC. In spite of resource limitations, our embedded zone MPC manages to achieve comparable performance of that of the full-version zone MPC implemented in a 64-bit desktop for scenarios with/without meal-disturbance compensations. Metrics for performance comparison included median percent time in the euglycemic ([70, 180] mg/dL range) of 84.3% vs. 83.1% for announced meals, with an equivalence test yielding p = 0.0013 and 66.2% vs. 66.0% for unannounced meals with p = 0.0028.

Pharmaceutical-based entrainment of circadian phase via nonlinear model predictive control.

The widespread adoption of closed-loop control in systems biology has resulted from improvements in sensors, computing, actuation, and the discovery of alternative sites of targeted drug delivery. Most control algorithms for circadian phase resetting exploit light inputs. However, recently identified small-molecule pharmaceuticals offer advantages in terms of invasiveness and potency of actuation. Herein, we develop a systematic method to control the phase of biological oscillations motivated by the recently identified small molecule circadian pharmaceutical KL001. The model-based control architecture exploits an infinitesimal parametric phase response curve (ipPRC) that is used to predict the effect of control inputs on future phase trajectories of the oscillator. The continuous time optimal control policy is first derived for phase resetting, based on the ipPRC and Pontryagin's maximum principle. Owing to practical challenges in implementing a continuous time optimal control policy, we investigate the effect of implementing the continuous time policy in a sampled time format. Specifically, we provide bounds on the errors incurred by the physiologically tractable sampled time control law. We use these results to select directions of resetting (i.e. phase advance or delay), sampling intervals, and prediction horizons for a nonlinear model predictive control (MPC) algorithm for phase resetting. The potential of this ipPRC-informed pharmaceutical nonlinear MPC is then demonstrated in silico using real-world scenarios of jet lag or rotating shift work.

A New Animal Model of Insulin-Glucose Dynamics in the Intraperitoneal Space Enhances Closed-Loop Control Performance.

Current artificial pancreas systems (AP) operate via subcutaneous (SC) glucose sensing and SC insulin delivery. Due to slow diffusion and transport dynamics across the interstitial space, even the most sophisticated control algorithms in on-body AP systems cannot react fast enough to maintain tight glycemic control under the effect of exogenous glucose disturbances caused by ingesting meals or performing physical activity. Recent efforts made towards the development of an implantable AP have explored the utility of insulin infusion in the intraperitoneal (IP) space: a region within the abdominal cavity where the insulin-glucose kinetics are observed to be much more rapid than the SC space. In this paper, a series of canine experiments are used to determine the dynamic association between IP insulin boluses and plasma glucose levels. Data from these experiments are employed to construct a new mathematical model and to formulate a closed-loop control strategy to be deployed on an implantable AP. The potential of the proposed controller is demonstrated via in-silico experiments on an FDA-accepted benchmark cohort: the proposed design significantly outperforms a previous controller designed using artificial data (time in clinically acceptable glucose range: 97.3±1.5% vs. 90.1±5.6%). Furthermore, the robustness of the proposed closed-loop system to delays and noise in the measurement signal (for example, when glucose is sensed subcutaneously) and deleterious glycemic changes (such as sudden glucose decline due to physical activity) is investigated. The proposed model based on experimental canine data leads to the generation of more effective control algorithms and is a promising step towards fully automated and implantable artificial pancreas systems.

Highly Accurate and Efficient Data-Driven Methods for Genotype Imputation.

High-throughput sequencing techniques have generated massive quantities of genotype data. Haplotype phasing has proven to be a useful and effective method for analyzing these data. However, the quality of phasing is undermined due to missing information. Imputation provides an effective means of improving the underlying genotype information. For model organisms, imputation can rely on an available reference genotype panel and a physical or genetic map. For non-model organisms, which often do not have a genotype panel, it is important to design an imputation technique that does not rely on reference data. Here, we present Accurate Data-Driven Imputation Technique (ADDIT), which is composed of two data-driven algorithms capable of handling data generated from model and non-model organisms. The non-model variant of ADDIT (referred to as ADDIT-NM) employs statistical inference methods to impute missing genotypes, whereas the model variant (referred to as ADDIT-M) leverages a supervised learning-based approach for imputation. We demonstrate that both variants of ADDIT are more accurate, faster, and require less memory than leading state-of-the-art imputation tools using model (human) and non-model (maize, apple, and grape) genotype data. Software Availability: The source code of ADDIT and test data sets are available at https://github.com/NDBL/ADDIT.

HECIL: A Hybrid Error Correction Algorithm for Long Reads with Iterative Learning.

Second-generation DNA sequencing techniques generate short reads that can result in fragmented genome assemblies. Third-generation sequencing platforms mitigate this limitation by producing longer reads that span across complex and repetitive regions. However, the usefulness of such long reads is limited because of high sequencing error rates. To exploit the full potential of these longer reads, it is imperative to correct the underlying errors. We propose HECIL-Hybrid Error Correction with Iterative Learning-a hybrid error correction framework that determines a correction policy for erroneous long reads, based on optimal combinations of decision weights obtained from short read alignments. We demonstrate that HECIL outperforms state-of-the-art error correction algorithms for an overwhelming majority of evaluation metrics on diverse, real-world data sets including E. coli, S. cerevisiae, and the malaria vector mosquito A. funestus. Additionally, we provide an optional avenue of improving the performance of HECIL's core algorithm by introducing an iterative learning paradigm that enhances the correction policy at each iteration by incorporating knowledge gathered from previous iterations via data-driven confidence metrics assigned to prior corrections.

Event-Triggered Model Predictive Control for Embedded Artificial Pancreas Systems.

OBJECTIVE: The development of artificial pancreas (AP) technology for deployment in low-energy, embedded devices is contingent upon selecting an efficient control algorithm for regulating glucose in people with type 1 diabetes mellitus. In this paper, we aim to lower the energy consumption of the AP by reducing controller updates, that is, the number of times the decision-making algorithm is invoked to compute an appropriate insulin dose. METHODS: Physiological insights into glucose management are leveraged to design an event-triggered model predictive controller (MPC) that operates efficiently, without compromising patient safety. The proposed event-triggered MPC is deployed on a wearable platform. Its robustness to latent hypoglycemia, model mismatch, and meal misinformation is tested, with and without meal announcement, on the full version of the US-FDA accepted UVA/Padova metabolic simulator. RESULTS: The event-based controller remains on for 18 h of 41 h in closed loop with unannounced meals, while maintaining glucose in 70-180 mg/dL for 25 h, compared to 27 h for a standard MPC controller. With meal announcement, the time in 70-180 mg/dL is almost identical, with the controller operating a mere 25.88% of the time in comparison with a standard MPC. CONCLUSION: A novel control architecture for AP systems enables safe glycemic regulation with reduced processor computations. SIGNIFICANCE: Our proposed framework integrated seamlessly with a wide variety of popular MPC variants reported in AP research, customizes tradeoff between glycemic regulation and efficacy according to prior design specifications, and eliminates judicious prior selection of controller sampling times.

Intraperitoneal insulin delivery provides superior glycaemic regulation to subcutaneous insulin delivery in model predictive control-based fully-automated artificial pancreas in patients with type 1 diabetes: a pilot study.

AIMS: To compare intraperitoneal (IP) to subcutaneous (SC) insulin delivery in an artificial pancreas (AP). RESEARCH DESIGN AND METHODS: Ten adults with type 1 diabetes participated in a non-randomized, non-blinded sequential AP study using the same SC glucose sensing and Zone Model Predictive Control (ZMPC) algorithm adjusted for insulin clearance. On first admission, subjects underwent closed-loop control with SC delivery of a fast-acting insulin analogue for 24 hours. Following implantation of a DiaPort IP insulin delivery system, the identical 24-hour trial was performed with IP regular insulin delivery. The clinical protocol included 3 unannounced meals with 70, 40 and 70 g carbohydrate, respectively. Primary endpoint was time spent with blood glucose (BG) in the range of 80 to 140 mg/dL (4.4-7.7 mmol/L). RESULTS: Percent of time spent within the 80 to 140 mg/dL range was significantly higher for IP delivery than for SC delivery: 39.8 ± 7.6 vs 25.6 ± 13.1 ( P = .03). Mean BG (mg/dL) and percent of time spent within the broader 70 to 180 mg/dL range were also significantly better for IP insulin: 151.0 ± 11.0 vs 190.0 ± 31.0 ( P = .004) and 65.7 ± 9.2 vs 43.9 ± 14.7 ( P = .001), respectively. Superiority of glucose control with IP insulin came from the reduced time spent in hyperglycaemia (>180 mg/dL: 32.4 ± 8.9 vs 53.5 ± 17.4, P = .014; >250 mg/dL: 5.9 ± 5.6 vs 23.0 ± 11.3, P = .0004). Higher daily doses of insulin (IU) were delivered with the IP route (43.7 ± 0.1 vs 32.3 ± 0.1, P < .001) with no increased percent time spent <70 mg/dL (IP: 2.5 ± 2.9 vs SC: 4.1 ± 5.3, P = .42). CONCLUSIONS: Glycaemic regulation with fully-automated AP delivering IP insulin was superior to that with SC insulin delivery. This pilot study provides proof-of-concept for an AP system combining a ZMPC algorithm with IP insulin delivery.

Nonlinear Model Predictive Control For Circadian Entrainment Using Small-Molecule Pharmaceuticals.

Recent in vitro studies have identified small-molecule pharmaceuticals effecting dose-dependent changes in the mammalian circadian clock, providing a novel avenue for control. Most studies employ light for clock control, however, pharmaceuticals are advantageous for clock manipulation through reduced invasiveness. In this paper, we employ a mechanistic model to predict the phase dynamics of the mammalian circadian oscillator under the effect of the pharmaceutical under investigation. These predictions are used to inform a constrained model predictive controller (MPC) to compute appropriate dosing for clock re-entrainment. Constraints in the formulation of the MPC problem arise from variation in the phase response curves (PRCs) describing drug effects, and are in many cases non-intuitive owing to the nonlinearity of oscillator phase response effects. We demonstrate through in-silico experiments that it is imperative to tune the MPC parameters based on the drug-specific PRC for optimal phase manipulation.

Correcting hypothalamic-pituitary-adrenal axis dysfunction using observer-based explicit nonlinear model predictive control.

The hypothalamic-pituitary-adrenal (HPA) axis is critical in maintaining homeostasis under physical and psychological stress by modulating cortisol levels in the body. Dysregulation of cortisol levels is linked to numerous stress-related disorders. In this paper, an automated treatment methodology is proposed, employing a variant of nonlinear model predictive control (NMPC), called explicit MPC (EMPC). The controller is informed by an unknown input observer (UIO), which estimates various hormonal levels in the HPA axis system in conjunction with the magnitude of the stress applied on the body, based on measured concentrations of adreno-corticotropic hormones (ACTH). The proposed closed-loop control strategy is tested on multiple in silico patients and the effectiveness of the controller performance is demonstrated.

Model-based design of experiments for cellular processes.

Model-based design of experiments (MBDOE) assists in the planning of highly effective and efficient experiments. Although the foundations of this field are well-established, the application of these techniques to understand cellular processes is a fertile and rapidly advancing area as the community seeks to understand ever more complex cellular processes and systems. This review discusses the MBDOE paradigm along with applications and challenges within the context of cellular processes and systems. It also provides a brief tutorial on Fisher information matrix (FIM)-based and Bayesian experiment design methods along with an overview of existing software packages and computational advances that support MBDOE application and adoption within the Systems Biology community. As cell-based products and biologics progress into the commercial sector, it is anticipated that MBDOE will become an essential practice for design, quality control, and production.

An integrated pathway system modeling of Saccharomyces cerevisiae HOG pathway: a Petri net based approach.

Biochemical networks comprise many diverse components and interactions between them. It has intracellular signaling, metabolic and gene regulatory pathways which are highly integrated and whose responses are elicited by extracellular actions. Previous modeling techniques mostly consider each pathway independently without focusing on the interrelation of these which actually functions as a single system. In this paper, we propose an approach of modeling an integrated pathway using an event-driven modeling tool, i.e., Petri nets (PNs). PNs have the ability to simulate the dynamics of the system with high levels of accuracy. The integrated set of signaling, regulatory and metabolic reactions involved in Saccharomyces cerevisiae's HOG pathway has been collected from the literature. The kinetic parameter values have been used for transition firings. The dynamics of the system has been simulated and the concentrations of major biological species over time have been observed. The phenotypic characteristics of the integrated system have been investigated under two conditions, viz., under the absence and presence of osmotic pressure. The results have been validated favorably with the existing experimental results. We have also compared our study with the study of idFBA (Lee et al., PLoS Comput Biol 4:e1000-e1086, 2008) and pointed out the differences between both studies. We have simulated and monitored concentrations of multiple biological entities over time and also incorporated feedback inhibition by Ptp2 which has not been included in the idFBA study. We have concluded that our study is the first to the best of our knowledge to model signaling, metabolic and regulatory events in an integrated form through PN model framework. This study is useful in computational simulation of system dynamics for integrated pathways as there are growing evidences that the malfunctioning of the interplay among these pathways is associated with disease.