Data-Driven Linear Quadratic Optimization for Controller Synthesis With Structural Constraints.

For various typical cases and situations where the formulation results in an optimal control problem, the linear quadratic regulator (LQR) approach and its variants continue to be highly attractive. In certain scenarios, it can happen that some prescribed structural constraints on the gain matrix would arise. Consequently then, the algebraic Riccati equation (ARE) is no longer applicable in a straightforward way to obtain the optimal solution. This work presents a rather effective alternative optimization approach based on gradient projection. The utilized gradient is obtained through a data-driven methodology, and then projected onto applicable constrained hyperplanes. Essentially, this projection gradient determines a direction of progression and computation for the gain matrix update with a decreasing functional cost; and then the gain matrix is further refined in an iterative framework. With this formulation, a data-driven optimization algorithm is summarized for controller synthesis with structural constraints. This data-driven approach has the key advantage that it avoids the necessity of precise modeling which is always required in the classical model-based counterpart; and thus the approach can additionally accommodate various model uncertainties. Illustrative examples are also provided in the work to validate the theoretical results.

Adaptive Safe Reinforcement Learning With Full-State Constraints and Constrained Adaptation for Autonomous Vehicles.

High-performance learning-based control for the typical safety-critical autonomous vehicles invariably requires that the full-state variables are constrained within the safety region even during the learning process. To solve this technically critical and challenging problem, this work proposes an adaptive safe reinforcement learning (RL) algorithm that invokes innovative safety-related RL methods with the consideration of constraining the full-state variables within the safety region with adaptation. These are developed toward assuring the attainment of the specified requirements on the full-state variables with two notable aspects. First, thus, an appropriately optimized backstepping technique and the asymmetric barrier Lyapunov function (BLF) methodology are used to establish the safe learning framework to ensure system full-state constraints requirements. More specifically, each subsystem's control and partial derivative of the value function are decomposed with asymmetric BLF-related items and an independent learning part. Then, the independent learning part is updated to solve the Hamilton-Jacobi-Bellman equation through an adaptive learning implementation to attain the desired performance in system control. Second, with further Lyapunov-based analysis, it is demonstrated that safety performance is effectively doubly assured via a methodology of a constrained adaptation algorithm during optimization (which incorporates the projection operator and can deal with the conflict between safety and optimization). Therefore, this algorithm optimizes system control and ensures that the full set of state variables involved is always constrained within the safety region during the whole learning process. Comparison simulations and ablation studies are carried out on motion control problems for autonomous vehicles, which have verified superior performance with smaller variance and better convergence performance under uncertain circumstances. The effectiveness of the safe performance of overall system control with the proposed method accordingly has been verified.

Local Learning Enabled Iterative Linear Quadratic Regulator for Constrained Trajectory Planning.

Trajectory planning is one of the indispensable and critical components in robotics and autonomous systems. As an efficient indirect method to deal with the nonlinear system dynamics in trajectory planning tasks over the unconstrained state and control space, the iterative linear quadratic regulator (iLQR) has demonstrated noteworthy outcomes. In this article, a local-learning-enabled constrained iLQR algorithm is herein presented for trajectory planning based on hybrid dynamic optimization and machine learning. Rather importantly, this algorithm attains the key advantage of circumventing the requirement of system identification, and the trajectory planning task is achieved with a simultaneous refinement of the optimal policy and the neural network system in an iterative framework. The neural network can be designed to represent the local system model with a simple architecture, and thus it leads to a sample-efficient training pipeline. In addition, in this learning paradigm, the constraints of the general form that are typically encountered in trajectory planning tasks are preserved. Several illustrative examples on trajectory planning are scheduled as part of the test itinerary to demonstrate the effectiveness and significance of this work.

Learning Asynchronous Boolean Networks From Single-Cell Data Using Multiobjective Cooperative Genetic Programming.

Recent advances in high-throughput single-cell technologies provide new opportunities for computational modeling of gene regulatory networks (GRNs) with an unprecedented amount of gene expression data. Current studies on the Boolean network (BN) modeling of GRNs mostly depend on bulk time-series data and focus on the synchronous update scheme due to its computational simplicity and tractability. However, such synchrony is a strong and rarely biologically realistic assumption. In this study, we adopt the asynchronous update scheme instead and propose a novel framework called SgpNet to infer asynchronous BNs from single-cell data by formulating it into a multiobjective optimization problem. SgpNet aims to find BNs that can match the asynchronous state transition graph (STG) extracted from single-cell data and retain the sparsity of GRNs. To search the huge solution space efficiently, we encode each Boolean function as a tree in genetic programming and evolve all functions of a network simultaneously via cooperative coevolution. Besides, we develop a regulator preselection strategy in view of GRN sparsity to further enhance learning efficiency. An error threshold estimation heuristic is also proposed to ease tedious parameter tuning. SgpNet is compared with the state-of-the-art method on both synthetic data and experimental single-cell data. Results show that SgpNet achieves comparable inference accuracy, while it has far fewer parameters and eliminates artificial restrictions on the Boolean function structures. Furthermore, SgpNet can potentially scale to large networks via straightforward parallelization on multiple cores.

Finite-Horizon Optimal Control of Boolean Control Networks: A Unified Graph-Theoretical Approach.

This article investigates the finite-horizon optimal control (FHOC) problem of Boolean control networks (BCNs) from a graph theory perspective. We first formulate two general problems to unify various special cases studied in the literature: 1) the horizon length is a priori fixed and 2) the horizon length is unspecified but finite for given destination states. Notably, both problems can incorporate time-variant costs, which are rarely considered in existing work, and a variety of constraints. The existence of an optimal control sequence is analyzed under mild assumptions. Motivated by BCNs' finite state space and control space, we approach the two general problems intuitively and efficiently under a graph-theoretical framework. A weighted state transition graph and its time-expanded variants are developed, and the equivalence between the FHOC problem and the shortest-path (SP) problem in specific graphs is established rigorously. Two algorithms are developed to find the SP and construct the optimal control sequence for the two problems with reduced computational complexity, though technically, a classical SP algorithm in graph theory is sufficient for all problems. Compared with existing algebraic methods, our graph-theoretical approach can achieve state-of-the-art time efficiency while targeting the most general problems. Furthermore, our approach is the first one capable of solving Problem 2) with time-variant costs. Finally, a genetic network in the bacterium E. coli and a signaling network involved in human leukemia are used to validate the effectiveness of our approach. The results of two common tasks for both networks show that our approach can dramatically reduce the running time. Python implementation of our algorithms is available at GitHub https://github.com/ShuhuaGao/FHOC.

Infinite-Horizon Optimal Control of Switched Boolean Control Networks With Average Cost: An Efficient Graph-Theoretical Approach.

This study investigates the infinite-horizon optimal control (IHOC) problem for switched Boolean control networks with an average cost criterion. A primary challenge of this problem is the prohibitively high computational cost when dealing with large-scale networks. We attempt to develop a more efficient approach from a novel graph-theoretical perspective. First, a weighted directed graph structure called the optimal state transition graph (OSTG) is established, whose edges encode the optimal action for each admissible state transition between states reachable from a given initial state subject to various constraints. Then, we reduce the IHOC problem into a minimum-mean cycle (MMC) problem in the OSTG. Finally, we develop an algorithm that can quickly find a particular MMC by resorting to Karp's algorithm in the graph theory and construct an optimal switching control law based on state feedback. The time complexity analysis shows that our algorithm, albeit still running in exponential time, can outperform all the existing methods in terms of time efficiency. A 16-state-3-input signaling network in leukemia is used as a benchmark to test its effectiveness. Results show that the proposed graph-theoretical approach is much more computationally efficient and can reduce the running time dramatically: it runs hundreds or even thousands of times faster than the existing methods. The Python implementation of the algorithm is available at https://github.com/ShuhuaGao/sbcn_mmc.

Barrier Lyapunov Function-Based Safe Reinforcement Learning for Autonomous Vehicles With Optimized Backstepping.

Guaranteed safety and performance under various circumstances remain technically critical and practically challenging for the wide deployment of autonomous vehicles. Safety-critical systems in general, require safe performance even during the reinforcement learning (RL) period. To address this issue, a Barrier Lyapunov Function-based safe RL (BLF-SRL) algorithm is proposed here for the formulated nonlinear system in strict-feedback form. This approach appropriately arranges and incorporates the BLF items into the optimized backstepping control method to constrain the state-variables in the designed safety region during learning. Wherein, thus, the optimal virtual/actual control in every backstepping subsystem is decomposed with BLF items and also with an adaptive uncertain item to be learned, which achieves safe exploration during the learning process. Then, the principle of Bellman optimality of continuous-time Hamilton-Jacobi-Bellman equation in every backstepping subsystem is satisfied with independently approximated actor and critic under the framework of actor-critic through the designed iterative updating. Eventually, the overall system control is optimized with the proposed BLF-SRL method. It is furthermore noteworthy that the variance of the attained control performance under uncertainty is also reduced with the proposed method. The effectiveness of the proposed method is verified with two motion control problems for autonomous vehicles through appropriate comparison simulations.

Trajectory Generation by Chance-Constrained Nonlinear MPC With Probabilistic Prediction.

Continued great efforts have been dedicated toward high-quality trajectory generation based on optimization methods; however, most of them do not suitably and effectively consider the situation with moving obstacles; and more particularly, the future position of these moving obstacles in the presence of uncertainty within some possible prescribed prediction horizon. To cater to this rather major shortcoming, this work shows how a variational Bayesian Gaussian mixture model (vBGMM) framework can be employed to predict the future trajectory of moving obstacles; and then with this methodology, a trajectory generation framework is proposed which will efficiently and effectively address trajectory generation in the presence of moving obstacles, and incorporate the presence of uncertainty within a prediction horizon. In this work, the full predictive conditional probability density function (PDF) with mean and covariance is obtained and, thus, a future trajectory with uncertainty is formulated as a collision region represented by a confidence ellipsoid. To avoid the collision region, chance constraints are imposed to restrict the collision probability, and subsequently, a nonlinear model predictive control problem is constructed with these chance constraints. It is shown that the proposed approach is able to predict the future position of the moving obstacles effectively; and, thus, based on the environmental information of the probabilistic prediction, it is also shown that the timing of collision avoidance can be earlier than the method without prediction. The tracking error and distance to obstacles of the trajectory with prediction are smaller compared with the method without prediction.

Cooperative Circumnavigation Control of Networked Microsatellites.

This paper addresses the trajectory analysis, mission design, and control law for multiple microsatellites to cooperatively circumnavigate a host spacecraft. This cooperative circumnavigation (CCN) problem is defined to drive a group of networked microsatellites to a predefined planar ellipse concerning a host spacecraft while maintaining a geometric formation configuration. We first design several potential functions to guide the microsatellites to the given planar elliptical orbit with a proper radius. Next, the affine Laplacian matrix is introduced to characterize the desired formation shape of microsatellites. Based on the potential functions and the Laplacian matrix, a CCN control law is finally proposed. Then, the simulation results of eight microsatellites with earth-orbiting mission scenarios are given, where the natural trajectory motion is incorporated which consumes nearly zero-fuel.

Context-dependent DNA coding with redundancy and introns.

Deoxyribonucleic acid (DNA) coding methods determine the meaning of a certain character in individual chromosomes by the characters surrounding it. The meaning of each character is context dependent, not position dependent. Although position-dependent coding is most commonly used in genetic algorithms (GAs), a context-dependent coding formation is in fact more closer to the natural DNA chromosome. With the context dependency, the DNA coding methods allow intron parts, redundancy, and variable string length in encoded strings while remaining compatible with the standard genetic operations. This paper tries to explicitly explore the influence of those special features of the DNA coding scheme. Two fundamental DNA coding methods (with and without the use of introns) are constructed and compared with the integer coding method, which lacks the features of interest. The performance of the proposed DNA coding methods is analyzed through the robot soccer role assignment problem. The context-dependent coding exhibits the advantages in handling the negative effect of epistasis. The redundancy and intron parts are helpful in preventing useful schemata from disruption and in increasing the population diversity. The variable length of the individual string enables GAs to evolve both the size and the structure of the fuzzy rule base.

Content-Driven Associative Memories for Color Image Patterns.

This paper presents a novel content-driven associative memory (CDAM) to associate large-scale color images based on the subjects that represent the images' content. Compared to traditional associative memories, CDAM inherits their tolerance to random noise in images and possesses greater robustness against correlated noise that distorts an image's spatial contextual structure. A three-layer recurrent neural tensor network (RNTN) is designed as the network model of CDAM. Multiple salient objects detection algorithm and partial radial basis function (PRBF) kernel are proposed for subject determination and content-driven association, respectively. Convergence of the RNTN is analyzed based on the properties of PRBF kernels. Extensive comparative experiment results are provided to verify the CDAM's efficiency, robustness, and accuracy.

Observer-based H infinity control for T-S fuzzy systems with time delay: delay-dependent design method.

This correspondence studies the problem of observer-based H infinity control for time-delay Takagi-Sugeno (T-S) fuzzy systems. It provides a delay-dependent linear matrix inequality (LMI)-based method for the control design. It is known that the key important problem in the literature, even for delay-independent case, lies in the difficulty of decoupling matrix variables in corresponding matrix inequalities. This correspondence suggests a decoupling technique for solving matrix inequalities with coupled variables, and provides an LMI-based algorithm by adopting the idea of the cone complementarity problem. The derivation relies on the appropriate choice of Lyaponuv-Krasovskii functionals which incorporate the intersections among local systems. Illustrative examples are given to show the effectiveness of the present delay-dependent result.

Reference Adaptation for Robots in Physical Interactions With Unknown Environments.

In this paper, we propose a method of reference adaptation for robots in physical interactions with unknown environments. A cost function is constructed to describe the interaction performance, which combines trajectory tracking error and interaction force between the robot and the environment. It is minimized by the proposed reference adaptation based on trajectory parametrization and iterative learning. An adaptive impedance control is developed to make the robot be governed by the target impedance model. Simulation and experiment studies are conducted to verify the effectiveness of the proposed method.

H(infinity) output tracking control for nonlinear systems via T-S fuzzy model approach.

This paper studies the problem of H(infinity) output tracking control for nonlinear time-delay systems using Takagi-Sugeno (T-S) fuzzy model approach. An LMI-based design method is proposed for achieving the output tracking purpose. Illustrative examples are given to show the effectiveness of the present results.

Nonlinear control of a dynamic model of HIV-1.

Highly active antiretroviral therapy (HAART) reduces the viral burden in human immunodeficiency virus type 1 (HIV-1) infected patients. The paper addresses the problem of controlling the predator-prey like model of the interaction among CD4+ T-cell, CD8+ T-cell, and HIV-1 by an external drug agency. By exploring the dynamic properties of the system, the original system is first regrouped into two subsystems, then a nonlinear global controller is presented by designing two controllers over two complementary zones: a local controller on a finite region and a global controller over its complement. The local controller is designed to guarantee nonnegativty, and avoids the problem of control singularity within the neighborhood of the origin omega. The complementary controller is designed via backstepping for both subsystems over the complementary region. The closed-loop system is globally stable at nominal values through the introduction of a novel bridging virtual control, and the resulting controller is singularity free and guarantees nonnegativity. In this paper, simulations are conducted in discrete-time with sampling time Ts to show the effectiveness of the proposed method.

Output feedback control of a class of discrete MIMO nonlinear systems with triangular form inputs.

In this paper, adaptive neural network (NN) control is investigated for a class of discrete-time multi-input-multi-output (MIMO) nonlinear systems with triangular form inputs. Each subsystem of the MIMO system is in strict feedback form. First, through two phases of coordinate transformation, the MIMO system is transformed into input-output representation with the triangular form input structure unchanged. By using high-order neural networks (HONNs) as the emulators of the desired controls, effective output feedback adaptive control is developed using backstepping. The closed-loop system is proved to be semiglobally uniformly ultimate bounded (SGUUB) by using Lyapunov method. The output tracking errors are guaranteed to converge into a compact set whose size is adjustable, and all the other signals in the closed-loop system are proved to be bounded. Simulation results show the effectiveness of the proposed control scheme.

Geometrical interpretation and architecture selection of MLP.

A geometrical interpretation of the multilayer perceptron (MLP) is suggested in this paper. Some general guidelines for selecting the architecture of the MLP, i.e., the number of the hidden neurons and the hidden layers, are proposed based upon this interpretation and the controversial issue of whether four-layered MLP is superior to the three-layered MLP is also carefully examined.

Practical adaptive neural control of nonlinear systems with unknown time delays.

Practical adaptive neural control is presented for a class of nonlinear systems with unknown time delays in strict-feedback form. Using appropriate Lyapunov-Krasovskii functionals, the unknown time delays are compensated for. Controller singularity problems are solved by practical neural network control. A novel differentiable control function is provided such that the practical design can be carried out in the decoupled backstepping design. It is proved that the proposed design method is able to guarantee semi-global uniform ultimate boundedness of all the signals in the closed-loop system, and the tracking error is proven to converge to a small neighborhood of the origin.

Optimal critic learning for robot control in time-varying environments.

In this paper, optimal critic learning is developed for robot control in a time-varying environment. The unknown environment is described as a linear system with time-varying parameters, and impedance control is employed for the interaction control. Desired impedance parameters are obtained in the sense of an optimal realization of the composite of trajectory tracking and force regulation. Q -function-based critic learning is developed to determine the optimal impedance parameters without the knowledge of the system dynamics. The simulation results are presented and compared with existing methods, and the efficacy of the proposed method is verified.

A unified quadratic-programming-based dynamical system approach to joint torque optimization of physically constrained redundant manipulators.

In this paper, for joint torque optimization of redundant manipulators subject to physical constraints, we show that velocity-level and acceleration-level redundancy-resolution schemes both can be formulated as a quadratic programming (QP) problem subject to equality and inequality/bound constraints. To solve this QP problem online, a primal-dual dynamical system solver is further presented based on linear variational inequalities. Compared to previous researches, the presented QP-solver has simple piecewise-linear dynamics, does not entail real-time matrix inversion, and could also provide joint-acceleration information for manipulator torque control in the velocity-level redundancy-resolution schemes. The proposed QP-based dynamical system approach is simulated based on the PUMA560 robot arm with efficiency and effectiveness demonstrated.