Data-Driven Linear Quadratic Optimization for Controller Synthesis With Structural Constraints.

For various typical cases and situations where the formulation results in an optimal control problem, the linear quadratic regulator (LQR) approach and its variants continue to be highly attractive. In certain scenarios, it can happen that some prescribed structural constraints on the gain matrix would arise. Consequently then, the algebraic Riccati equation (ARE) is no longer applicable in a straightforward way to obtain the optimal solution. This work presents a rather effective alternative optimization approach based on gradient projection. The utilized gradient is obtained through a data-driven methodology, and then projected onto applicable constrained hyperplanes. Essentially, this projection gradient determines a direction of progression and computation for the gain matrix update with a decreasing functional cost; and then the gain matrix is further refined in an iterative framework. With this formulation, a data-driven optimization algorithm is summarized for controller synthesis with structural constraints. This data-driven approach has the key advantage that it avoids the necessity of precise modeling which is always required in the classical model-based counterpart; and thus the approach can additionally accommodate various model uncertainties. Illustrative examples are also provided in the work to validate the theoretical results.

Adaptive Safe Reinforcement Learning With Full-State Constraints and Constrained Adaptation for Autonomous Vehicles.

High-performance learning-based control for the typical safety-critical autonomous vehicles invariably requires that the full-state variables are constrained within the safety region even during the learning process. To solve this technically critical and challenging problem, this work proposes an adaptive safe reinforcement learning (RL) algorithm that invokes innovative safety-related RL methods with the consideration of constraining the full-state variables within the safety region with adaptation. These are developed toward assuring the attainment of the specified requirements on the full-state variables with two notable aspects. First, thus, an appropriately optimized backstepping technique and the asymmetric barrier Lyapunov function (BLF) methodology are used to establish the safe learning framework to ensure system full-state constraints requirements. More specifically, each subsystem's control and partial derivative of the value function are decomposed with asymmetric BLF-related items and an independent learning part. Then, the independent learning part is updated to solve the Hamilton-Jacobi-Bellman equation through an adaptive learning implementation to attain the desired performance in system control. Second, with further Lyapunov-based analysis, it is demonstrated that safety performance is effectively doubly assured via a methodology of a constrained adaptation algorithm during optimization (which incorporates the projection operator and can deal with the conflict between safety and optimization). Therefore, this algorithm optimizes system control and ensures that the full set of state variables involved is always constrained within the safety region during the whole learning process. Comparison simulations and ablation studies are carried out on motion control problems for autonomous vehicles, which have verified superior performance with smaller variance and better convergence performance under uncertain circumstances. The effectiveness of the safe performance of overall system control with the proposed method accordingly has been verified.

Local Learning Enabled Iterative Linear Quadratic Regulator for Constrained Trajectory Planning.

Trajectory planning is one of the indispensable and critical components in robotics and autonomous systems. As an efficient indirect method to deal with the nonlinear system dynamics in trajectory planning tasks over the unconstrained state and control space, the iterative linear quadratic regulator (iLQR) has demonstrated noteworthy outcomes. In this article, a local-learning-enabled constrained iLQR algorithm is herein presented for trajectory planning based on hybrid dynamic optimization and machine learning. Rather importantly, this algorithm attains the key advantage of circumventing the requirement of system identification, and the trajectory planning task is achieved with a simultaneous refinement of the optimal policy and the neural network system in an iterative framework. The neural network can be designed to represent the local system model with a simple architecture, and thus it leads to a sample-efficient training pipeline. In addition, in this learning paradigm, the constraints of the general form that are typically encountered in trajectory planning tasks are preserved. Several illustrative examples on trajectory planning are scheduled as part of the test itinerary to demonstrate the effectiveness and significance of this work.

Barrier Lyapunov Function-Based Safe Reinforcement Learning for Autonomous Vehicles With Optimized Backstepping.

Guaranteed safety and performance under various circumstances remain technically critical and practically challenging for the wide deployment of autonomous vehicles. Safety-critical systems in general, require safe performance even during the reinforcement learning (RL) period. To address this issue, a Barrier Lyapunov Function-based safe RL (BLF-SRL) algorithm is proposed here for the formulated nonlinear system in strict-feedback form. This approach appropriately arranges and incorporates the BLF items into the optimized backstepping control method to constrain the state-variables in the designed safety region during learning. Wherein, thus, the optimal virtual/actual control in every backstepping subsystem is decomposed with BLF items and also with an adaptive uncertain item to be learned, which achieves safe exploration during the learning process. Then, the principle of Bellman optimality of continuous-time Hamilton-Jacobi-Bellman equation in every backstepping subsystem is satisfied with independently approximated actor and critic under the framework of actor-critic through the designed iterative updating. Eventually, the overall system control is optimized with the proposed BLF-SRL method. It is furthermore noteworthy that the variance of the attained control performance under uncertainty is also reduced with the proposed method. The effectiveness of the proposed method is verified with two motion control problems for autonomous vehicles through appropriate comparison simulations.

Learning Asynchronous Boolean Networks From Single-Cell Data Using Multiobjective Cooperative Genetic Programming.

Recent advances in high-throughput single-cell technologies provide new opportunities for computational modeling of gene regulatory networks (GRNs) with an unprecedented amount of gene expression data. Current studies on the Boolean network (BN) modeling of GRNs mostly depend on bulk time-series data and focus on the synchronous update scheme due to its computational simplicity and tractability. However, such synchrony is a strong and rarely biologically realistic assumption. In this study, we adopt the asynchronous update scheme instead and propose a novel framework called SgpNet to infer asynchronous BNs from single-cell data by formulating it into a multiobjective optimization problem. SgpNet aims to find BNs that can match the asynchronous state transition graph (STG) extracted from single-cell data and retain the sparsity of GRNs. To search the huge solution space efficiently, we encode each Boolean function as a tree in genetic programming and evolve all functions of a network simultaneously via cooperative coevolution. Besides, we develop a regulator preselection strategy in view of GRN sparsity to further enhance learning efficiency. An error threshold estimation heuristic is also proposed to ease tedious parameter tuning. SgpNet is compared with the state-of-the-art method on both synthetic data and experimental single-cell data. Results show that SgpNet achieves comparable inference accuracy, while it has far fewer parameters and eliminates artificial restrictions on the Boolean function structures. Furthermore, SgpNet can potentially scale to large networks via straightforward parallelization on multiple cores.

Infinite-Horizon Optimal Control of Switched Boolean Control Networks With Average Cost: An Efficient Graph-Theoretical Approach.

This study investigates the infinite-horizon optimal control (IHOC) problem for switched Boolean control networks with an average cost criterion. A primary challenge of this problem is the prohibitively high computational cost when dealing with large-scale networks. We attempt to develop a more efficient approach from a novel graph-theoretical perspective. First, a weighted directed graph structure called the optimal state transition graph (OSTG) is established, whose edges encode the optimal action for each admissible state transition between states reachable from a given initial state subject to various constraints. Then, we reduce the IHOC problem into a minimum-mean cycle (MMC) problem in the OSTG. Finally, we develop an algorithm that can quickly find a particular MMC by resorting to Karp's algorithm in the graph theory and construct an optimal switching control law based on state feedback. The time complexity analysis shows that our algorithm, albeit still running in exponential time, can outperform all the existing methods in terms of time efficiency. A 16-state-3-input signaling network in leukemia is used as a benchmark to test its effectiveness. Results show that the proposed graph-theoretical approach is much more computationally efficient and can reduce the running time dramatically: it runs hundreds or even thousands of times faster than the existing methods. The Python implementation of the algorithm is available at https://github.com/ShuhuaGao/sbcn_mmc.

Finite-Horizon Optimal Control of Boolean Control Networks: A Unified Graph-Theoretical Approach.

This article investigates the finite-horizon optimal control (FHOC) problem of Boolean control networks (BCNs) from a graph theory perspective. We first formulate two general problems to unify various special cases studied in the literature: 1) the horizon length is a priori fixed and 2) the horizon length is unspecified but finite for given destination states. Notably, both problems can incorporate time-variant costs, which are rarely considered in existing work, and a variety of constraints. The existence of an optimal control sequence is analyzed under mild assumptions. Motivated by BCNs' finite state space and control space, we approach the two general problems intuitively and efficiently under a graph-theoretical framework. A weighted state transition graph and its time-expanded variants are developed, and the equivalence between the FHOC problem and the shortest-path (SP) problem in specific graphs is established rigorously. Two algorithms are developed to find the SP and construct the optimal control sequence for the two problems with reduced computational complexity, though technically, a classical SP algorithm in graph theory is sufficient for all problems. Compared with existing algebraic methods, our graph-theoretical approach can achieve state-of-the-art time efficiency while targeting the most general problems. Furthermore, our approach is the first one capable of solving Problem 2) with time-variant costs. Finally, a genetic network in the bacterium E. coli and a signaling network involved in human leukemia are used to validate the effectiveness of our approach. The results of two common tasks for both networks show that our approach can dramatically reduce the running time. Python implementation of our algorithms is available at GitHub https://github.com/ShuhuaGao/FHOC.

Trajectory Generation by Chance-Constrained Nonlinear MPC With Probabilistic Prediction.

Continued great efforts have been dedicated toward high-quality trajectory generation based on optimization methods; however, most of them do not suitably and effectively consider the situation with moving obstacles; and more particularly, the future position of these moving obstacles in the presence of uncertainty within some possible prescribed prediction horizon. To cater to this rather major shortcoming, this work shows how a variational Bayesian Gaussian mixture model (vBGMM) framework can be employed to predict the future trajectory of moving obstacles; and then with this methodology, a trajectory generation framework is proposed which will efficiently and effectively address trajectory generation in the presence of moving obstacles, and incorporate the presence of uncertainty within a prediction horizon. In this work, the full predictive conditional probability density function (PDF) with mean and covariance is obtained and, thus, a future trajectory with uncertainty is formulated as a collision region represented by a confidence ellipsoid. To avoid the collision region, chance constraints are imposed to restrict the collision probability, and subsequently, a nonlinear model predictive control problem is constructed with these chance constraints. It is shown that the proposed approach is able to predict the future position of the moving obstacles effectively; and, thus, based on the environmental information of the probabilistic prediction, it is also shown that the timing of collision avoidance can be earlier than the method without prediction. The tracking error and distance to obstacles of the trajectory with prediction are smaller compared with the method without prediction.

Cooperative Circumnavigation Control of Networked Microsatellites.

This paper addresses the trajectory analysis, mission design, and control law for multiple microsatellites to cooperatively circumnavigate a host spacecraft. This cooperative circumnavigation (CCN) problem is defined to drive a group of networked microsatellites to a predefined planar ellipse concerning a host spacecraft while maintaining a geometric formation configuration. We first design several potential functions to guide the microsatellites to the given planar elliptical orbit with a proper radius. Next, the affine Laplacian matrix is introduced to characterize the desired formation shape of microsatellites. Based on the potential functions and the Laplacian matrix, a CCN control law is finally proposed. Then, the simulation results of eight microsatellites with earth-orbiting mission scenarios are given, where the natural trajectory motion is incorporated which consumes nearly zero-fuel.

Content-Driven Associative Memories for Color Image Patterns.

This paper presents a novel content-driven associative memory (CDAM) to associate large-scale color images based on the subjects that represent the images' content. Compared to traditional associative memories, CDAM inherits their tolerance to random noise in images and possesses greater robustness against correlated noise that distorts an image's spatial contextual structure. A three-layer recurrent neural tensor network (RNTN) is designed as the network model of CDAM. Multiple salient objects detection algorithm and partial radial basis function (PRBF) kernel are proposed for subject determination and content-driven association, respectively. Convergence of the RNTN is analyzed based on the properties of PRBF kernels. Extensive comparative experiment results are provided to verify the CDAM's efficiency, robustness, and accuracy.

Reference Adaptation for Robots in Physical Interactions With Unknown Environments.

In this paper, we propose a method of reference adaptation for robots in physical interactions with unknown environments. A cost function is constructed to describe the interaction performance, which combines trajectory tracking error and interaction force between the robot and the environment. It is minimized by the proposed reference adaptation based on trajectory parametrization and iterative learning. An adaptive impedance control is developed to make the robot be governed by the target impedance model. Simulation and experiment studies are conducted to verify the effectiveness of the proposed method.

Optimal critic learning for robot control in time-varying environments.

In this paper, optimal critic learning is developed for robot control in a time-varying environment. The unknown environment is described as a linear system with time-varying parameters, and impedance control is employed for the interaction control. Desired impedance parameters are obtained in the sense of an optimal realization of the composite of trajectory tracking and force regulation. Q -function-based critic learning is developed to determine the optimal impedance parameters without the knowledge of the system dynamics. The simulation results are presented and compared with existing methods, and the efficacy of the proposed method is verified.

On computation of stabilizing loop gain and delay ranges for bi-proper delay systems.

A graphical method for exactly computing the stabilizing loop gain and delay ranges was proposed [Le BN, Wang Q-G, Lee T-H. Development of D-decomposition method for computing stabilizing gain ranges for general delay systems. J Process Control 2012] for a strictly proper process by determining the boundary functions which may change system׳s stability. A bi-proper process is rare but causes great complications for the method, due to the new phenomena that do not exist for a strictly proper process, such as a non-zero gain at infinity frequency, which may cause infinite intersections of boundary functions within a finite delay range. This paper addresses such a kind of processes and develops a general method that can produce the exact and complete set of the loop gain and delay for closed-loop stabilization, which is hard to find with analytical methods.

Dynamically weighted ensemble classification for non-stationary EEG processing.

OBJECTIVE: The non-stationary nature of EEG poses a major challenge to robust operation of brain-computer interfaces (BCIs). The objective of this paper is to propose and investigate a computational method to address non-stationarity in EEG classification. APPROACH: We developed a novel dynamically weighted ensemble classification (DWEC) framework whereby an ensemble of multiple classifiers are trained on clustered features. The decisions from these multiple classifiers are dynamically combined based on the distances of the cluster centres to each test data sample being classified. MAIN RESULTS: The clusters of the feature space from the second session spanned a different space compared to the clusters of the feature space from the first session which highlights the processes of session-to-session non-stationarity. The session-to-session performance of the proposed DWEC method was evaluated on two datasets. The results on publicly available BCI Competition IV dataset 2A yielded a significantly higher mean accuracy of 81.48% compared to 75.9% from the baseline support vector machine (SVM) classifier without dynamic weighting. Results on the data collected from our twelve in-house subjects yielded a significantly higher mean accuracy of 73% compared to 69.4% from the baseline SVM classifier without dynamic weighting. SIGNIFICANCE: The cluster based analysis provides insight into session-to-session non-stationarity in EEG data. The results demonstrate the effectiveness of the proposed method in addressing non-stationarity in EEG data for the operation of a BCI.

SVD-based Preisach hysteresis identification and composite control of piezo actuators.

In this paper, the singular value decomposition (SVD) based identification and compensation of the hysteretic phenomenon in piezo actuators are addressed using a Preisach model. First, this paper presents an SVD-based least squares algorithm and a revision approach of the identification through updating the SVD. With the identified parameters and a log of the memory curve, a Preisach-based inversion compensator is constructed which is complemented with a feedback controller to address the inevitable and residual modeling errors. Experimental results are furnished for both the identification and compensation approaches. The Preisach-based feedforward controller significantly improves the tracking performance and reduces the root-mean-square (RMS) tracking error of a PID controller by 76.7% and 89% at 1 Hz and 25 Hz, respectively. With the proposed composite controller, the percent-RMS errors at 1 Hz and 25 Hz are reduced to 0.035% and 0.31%, respectively.

Data-based identification and control of nonlinear systems via piecewise affine approximation.

The piecewise affine (PWA) model represents an attractive model structure for approximating nonlinear systems. In this paper, a procedure for obtaining the PWA autoregressive exogenous (ARX) (autoregressive systems with exogenous inputs) models of nonlinear systems is proposed. Two key parameters defining a PWARX model, namely, the parameters of locally affine subsystems and the partition of the regressor space, are estimated, the former through a least-squares-based identification method using multiple models, and the latter using standard procedures such as neural network classifier or support vector machine classifier. Having obtained the PWARX model of the nonlinear system, a controller is then derived to control the system for reference tracking. Both simulation and experimental studies show that the proposed algorithm can indeed provide accurate PWA approximation of nonlinear systems, and the designed controller provides good tracking performance.

Adaptive output feedback NN control of a class of discrete-time MIMO nonlinear systems with unknown control directions.

In this paper, adaptive neural network (NN) control is investigated for a class of block triangular multiinput-multioutput nonlinear discrete-time systems with each subsystem in pure-feedback form with unknown control directions. These systems are of couplings in every equation of each subsystem, and different subsystems may have different orders. To avoid the noncausal problem in the control design, the system is transformed into a predictor form by rigorous derivation. By exploring the properties of the block triangular form, implicit controls are developed for each subsystem such that the couplings of inputs and states among subsystems have been completely decoupled. The radial basis function NN is employed to approximate the unknown control. Each subsystem achieves a semiglobal uniformly ultimately bounded stability with the proposed control, and simulation results are presented to demonstrate its efficiency.

Adaptive neural control for output feedback nonlinear systems using a barrier Lyapunov function.

In this brief, adaptive neural control is presented for a class of output feedback nonlinear systems in the presence of unknown functions. The unknown functions are handled via on-line neural network (NN) control using only output measurements. A barrier Lyapunov function (BLF) is introduced to address two open and challenging problems in the neuro-control area: 1) for any initial compact set, how to determine a priori the compact superset, on which NN approximation is valid; and 2) how to ensure that the arguments of the unknown functions remain within the specified compact superset. By ensuring boundedness of the BLF, we actively constrain the argument of the unknown functions to remain within a compact superset such that the NN approximation conditions hold. The semiglobal boundedness of all closed-loop signals is ensured, and the tracking error converges to a neighborhood of zero. Simulation results demonstrate the effectiveness of the proposed approach.

Microdispensing system via the contacting method.

This paper presents the development of a microdispensing system based on a contacting method, with an aim to lowering production and maintenance cost. The liquid, to be dispensed, is brought into contact with the work piece, thus dispensing a droplet by making use of the adhesion force between the liquid and the work piece. A piezoelectric actuator is employed as the drive for the system to achieve high precision. The control of the system is accomplished with a PID controller; controlling the dispensing process and trajectory tracking.

Decoupled tracking and thermal monitoring of non-stationary targets.

Fault diagnosis and predictive maintenance address pertinent economic issues relating to production systems as an efficient technique can continuously monitor key health parameters and trigger alerts when critical changes in these variables are detected, before they lead to system failures and production shutdowns. In this paper, we present a decoupled tracking and thermal monitoring system which can be used on non-stationary targets of closed systems such as machine tools. There are three main contributions from the paper. First, a vision component is developed to track moving targets under a monitor. Image processing techniques are used to resolve the target location to be tracked. Thus, the system is decoupled and applicable to closed systems without the need for a physical integration. Second, an infrared temperature sensor with a built-in laser for locating the measurement spot is deployed for non-contact temperature measurement of the moving target. Third, a predictive motion control system holds the thermal sensor and follows the moving target efficiently to enable continuous temperature measurement and monitoring.