Structure-Based Inverse Reinforcement Learning for Quantification of Biological Knowledge.

Gene regulatory networks (GRNs) play crucial roles in various cellular processes, including stress response, DNA repair, and the mechanisms involved in complex diseases such as cancer. Biologists are involved in most biological analyses. Thus, quantifying their policies reflected in available biological data can significantly help us to better understand these complex systems. The primary challenges preventing the utilization of existing machine learning, particularly inverse reinforcement learning techniques, to quantify biologists' knowledge are the limitations and huge amount of uncertainty in biological data. This paper leverages the network-like structure of GRNs to define expert reward functions that contain exponentially fewer parameters than regular reward models. Numerical experiments using mammalian cell cycle and synthetic gene-expression data demonstrate the superior performance of the proposed method in quantifying biologists' policies.

Optimal Recursive Expert-Enabled Inference in Regulatory Networks.

Accurate inference of biological systems, such as gene regulatory networks and microbial communities, is a key to a deep understanding of their underlying mechanisms. Despite several advances in the inference of regulatory networks in recent years, the existing techniques cannot incorporate expert knowledge into the inference process. Expert knowledge contains valuable biological information and is often reflected in available biological data, such as interventions made by biologists for treating diseases. Given the complexity of regulatory networks and the limitation of biological data, ignoring expert knowledge can lead to inaccuracy in the inference process. This paper models the regulatory networks using Boolean network with perturbation. We develop an expert-enabled inference method for inferring the unknown parameters of the network model using expert-acquired data. Given the availability of information about data-acquiring objectives and expert confidence, the proposed method optimally quantifies the expert knowledge along with the temporal changes in the data for the inference process. The numerical experiments investigate the performance of the proposed method using the well-known p53-MDM2 gene regulatory network.

Scalable Inverse Reinforcement Learning Through Multifidelity Bayesian Optimization.

Data in many practical problems are acquired according to decisions or actions made by users or experts to achieve specific goals. For instance, policies in the mind of biologists during the intervention process in genomics and metagenomics are often reflected in available data in these domains, or data in cyber-physical systems are often acquired according to actions/decisions made by experts/engineers for purposes, such as control or stabilization. Quantification of experts' policies through available data, which is also known as reward function learning, has been discussed extensively in the literature in the context of inverse reinforcement learning (IRL). However, most of the available techniques come short to deal with practical problems due to the following main reasons: 1) lack of scalability: arising from incapability or poor performance of existing techniques in dealing with large systems and 2) lack of reliability: coming from the incapability of the existing techniques to properly learn the optimal reward function during the learning process. Toward this, in this brief, we propose a multifidelity Bayesian optimization (MFBO) framework that significantly scales the learning process of a wide range of existing IRL techniques. The proposed framework enables the incorporation of multiple approximators and efficiently takes their uncertainty and computational costs into account to balance exploration and exploitation during the learning process. The proposed framework's high performance is demonstrated through genomics, metagenomics, and sets of random simulated problems.

Two-Stage Bayesian Optimization for Scalable Inference in State-Space Models.

State-space models (SSMs) are a rich class of dynamical models with a wide range of applications in economics, healthcare, computational biology, robotics, and more. Proper analysis, control, learning, and decision-making in dynamical systems modeled by SSMs depend on the accuracy of the inferred/learned model. Most of the existing inference techniques for SSMs are capable of dealing with very small systems, unable to be applied to most of the large-scale practical problems. Toward this, this article introduces a two-stage Bayesian optimization (BO) framework for scalable and efficient inference in SSMs. The proposed framework maps the original large parameter space to a reduced space, containing a small linear combination of the original space. This reduced space, which captures the most variability in the inference function (e.g., log likelihood or log a posteriori), is obtained by eigenvalue decomposition of the covariance of gradients of the inference function approximated by a particle filtering scheme. Then, an exponential reduction in the search space of parameters during the inference process is achieved through the proposed two-stage BO policy, where the solution of the first-stage BO policy in the reduced space specifies the search space of the second-stage BO in the original space. The proposed framework's accuracy and speed are demonstrated through several experiments, including real metagenomics data from a gut microbial community.

Graph-Based Bayesian Optimization for Large-Scale Objective-Based Experimental Design.

Design is an inseparable part of most scientific and engineering tasks, including real and simulation-based experimental design processes and parameter/hyperparameter tuning/optimization. Several model-based experimental design techniques have been developed for design in domains with partial available knowledge about the underlying process. This article focuses on a powerful class of model-based experimental design called the mean objective cost of uncertainty (MOCU). The MOCU-based techniques are objective-based, meaning that they take the main objective of the process into account during the experimental design process. However, the lack of scalability of MOCU-based techniques prevents their application to most practical problems, including large discrete or combinatorial spaces. To achieve a scalable objective-based experimental design, this article proposes a graph-based MOCU-based Bayesian optimization framework. The correlations among samples in the large design space are accounted for using a graph-based Gaussian process, and an efficient closed-form sequential selection is achieved through the well-known expected improvement policy. The proposed framework's performance is assessed through the structural intervention in gene regulatory networks, aiming to make the network away from the states associated with cancer.

Bayesian Optimization for Design of Multi-Actuator Soft Catheter Robots.

Catheter-based diagnosis and therapy have grown increasingly in recent years due to their improved clinical outcomes including decreased morbidity, shorter recovery time and minimally invasiveness compared to open surgeries. Although the scalability, customizability, and diversity of soft catheter robots are widely recognized, designers and roboticists still lack comprehensive techniques for modeling and designing them. This difficulty arises due to their continuum nature, which makes characterizing the properties and predicting a soft catheter's behavior challenging, complicating robot design tasks. In this paper, we propose modeling multi-actuator soft catheters to enable alignment with desired vessel shapes near the target area. We develop mathematical models to simulate the catheter's positioning due to the moments exerted by multiple pneumatic actuators along the catheter and use those models to compare optimization approaches that can achieve catheter alignment along a desired vessel shape. Specifically, our approach proposes finding the optimal geometric and material properties for a multi-actuator soft catheter robot using a bi-level optimization framework. The upper-level optimization process uses a modified Bayesian technique to seek the optimal geometric and material properties of the soft catheter, which minimize the deviance of the actuated catheter from a desired vessel shape, while the lower-level optimization process uses a gradient-based technique to obtain the actuator moments required to achieve that vessel shape. The results demonstrate the capability of our proposed multi-actuator soft catheter to align with the desired vessel shapes, and show that the proposed framework which is in the context of Bayesian optimization has the potential to expedite the design process.

Physics-Informed Modeling and Control of Multi-Actuator Soft Catheter Robots.

Catheter-based endovascular interventional procedures have become increasingly popular in recent years as they are less invasive and patients spend less time in the hospital with less recovery time and less pain. These advantages have led to a significant growth in the number of procedures that are performed annually. However, it is still challenging to position a catheter in a target vessel branch within the highly complicated and delicate vascular structure. In fact, vessel tortuosity and angulation, which cause difficulties in catheterization and reaching the target site, have been reported as the main causes of failure in endovascular procedures. Maneuverability of a catheter for intravascular navigation is a key to reaching the target area; ability of a catheter to move within the target vessel during trajectory tracking thus affects to a great extent the length and success of the procedure. To address this issue, this paper models soft catheter robots with multiple actuators and provides a time-dependent model for characterizing the dynamics of multi-actuator soft catheter robots. Built on this model, an efficient and scalable optimization-based framework is developed for guiding the catheter to pass through arteries and reach the target where an aneurysm is located. The proposed framework models the deflection of the multi-actuator soft catheter robot and develops a control strategy for movement of catheter along a desired trajectory. This provides a simulation-based framework for selection of catheters prior to endovascular catheterization procedures, assuring that given a fixed design, the catheter is able to reach the target location. The results demonstrate the benefits that can be achieved by design and control of catheters with multiple number of actuators for navigation into small vessels.