Learning to Perform Trajectory Generation From Low-Quality Demonstrations.

Human-robot skill transfer is an important means for robots to learn skills and has received more and more attention and research in recent years. Typically, to ensure effective skill transfer, a skill is demonstrated several times by a human, from which a robot learns the features contained in the demonstrations and reproduces the skill in a new environment. However, it is necessary to consider the cases such as errors in human demonstrations and sensor issues, resulting in imperfect demonstrations, unrelated data, information loss, and variations in the lengths and amplitudes of the demonstrations. Therefore, this brief proposes a new trajectory alignment and filtering method for extracting relatively useful information from multiple demonstrations. This method can be used in conjunction with most probabilistic movement learning methods (this brief uses probabilistic movement primitives (ProMPs) as an example) for learning from demonstrations (LfDs), so that the robot can eventually learn and generate trajectories for completing skills from multiple demonstrations of varying quality. The effectiveness of the proposed method is verified by simulation results.

An Efficient Matching Game Approach to Association Formation in UAV-Enabled Hierarchical Distributed Learning.

Distributed machine learning has emerged as a promising data processing technology for next-generation communication systems. It leverages the computational capabilities of local nodes to efficiently handle large datasets, creating highly accurate data-driven models for analysis and prediction purposes. However, the performance of distributed machine learning can be significantly hampered by communication bottlenecks and node dropouts. In this article, a novel unmanned aerial vehicle (UAV)-enabled hierarchical distributed learning architecture is proposed to support machine learning applications, e.g., regional monitoring. Multiple UAV receivers (URs) are introduced as wireless relays to improve the communication between the UAV transmitters (UTs) and the cloud server. Our objective is to identify the optimal UT-UR association to maximize the social welfare of the network, which is distinctly different from the existing works that focus on the unilateral profit-maximizing problem. We formulate a two-side many-to-one matching game to model the UT-UR association problem, and a two-phase many-to-one matching algorithm is designed to identify the stable matching. The validity of our proposed scheme is verified through in-depth numerical simulations.

Learning Robust Predictive Control: A Spatial-Temporal Game Theoretic Approach.

This article investigates robust predictive control problem for unknown dynamical systems. Since the dynamics unavailability restricts feasibility of model-driven methods, learning robust predictive control (LRPC) framework is developed from the aspect of time consistency. Under feedback-like control causality, the robust predictive control is then reconstructed as spatialbKKtemporal games, and we guarantee stability through time-consistent Nash equilibrium. For gradation clarity, our framework is specified as four-follow contents. First, multistep feedback-like control causality is drawn from time series analysis, and Takens' theorem provides theoretical support from steady-state property. Second, control problem is reconstructed as games, while performance and robustness partition the game into temporal nonzero-sum subgames and spatial zero-sum ones, respectively. Next, multistep reinforcement learning (RL) is designed to solve robust predictive control without system model. Convergence is proven through bounds analysis of oscillatory value functions, and properties of receding horizon are derived from time consistency. Finally, data-driven implementation is given with function approximation, and neural networks are chosen to approximate value functions and feedback-like causality. Weights are estimated with least squares errors. Numerical results verify the effectiveness.

Decentralized Control for Large-Scale Systems With Actuator Faults and External Disturbances: A Data-Driven Method.

This article investigates optimal control for a class of large-scale systems using a data-driven method. The existing control methods for large-scale systems in this context separately consider disturbances, actuator faults, and uncertainties. In this article, we build on such methods by proposing an architecture that accommodates simultaneous consideration of all of these effects, and an optimization index is designed for the control problem. This diversifies the class of large-scale systems amenable to optimal control. We first establish a min-max optimization index based on the zero-sum differential game theory. Then, by integrating all the Nash equilibrium solutions of the isolated subsystems, the decentralized zero-sum differential game strategy is obtained to stabilize the large-scale system. Meanwhile, by designing adaptive parameters, the impact of actuator failure on the system performance is eliminated. Afterward, an adaptive dynamic programming (ADP) method is utilized to learn the solution of the Hamilton-Jacobi-Isaac (HJI) equation, which does not need the prior knowledge of system dynamics. A rigorous stability analysis shows that the proposed controller asymptotically stabilizes the large-scale system. Finally, a multipower system example is adopted to illustrate the effectiveness of the proposed protocols.

Data-Based Predictive Control via Multistep Policy Gradient Reinforcement Learning.

In this article, a model-free predictive control algorithm for the real-time system is presented. The algorithm is data driven and is able to improve system performance based on multistep policy gradient reinforcement learning. By learning from the offline dataset and real-time data, the knowledge of system dynamics is avoided in algorithm design and application. Cooperative games of the multiplayer in time horizon are presented to model the predictive control as optimization problems of multiagent and guarantee the optimality of the predictive control policy. In order to implement the algorithm, neural networks are used to approximate the action-state value function and predictive control policy, respectively. The weights are determined by using the methods of weighted residual. Numerical results show the effectiveness of the proposed algorithm.

Output Consensus of Heterogeneous Linear Multiagent Systems With Directed Graphs via Adaptive Dynamic Event-Triggered Mechanism.

This article investigates the output consensus problem of heterogeneous linear multiagent systems under directed communication graphs. A novel adaptive dynamic event-triggered mechanism is proposed, intending to further save system resources consumed in communication between agents and controller update of agents themselves, and remove the assumption that the global information associated with the communication topology should be known in advance for the design of control parameters. Unlike the existing related adaptive event-triggered algorithms, the proposed algorithm could theoretically guarantee the existence of a strictly positive constant on the interevent time intervals for both communication between agents and controller update. Furthermore, it is shown by the simulation that the addition of a dynamic variable has the potential to further optimize the control cost when compared to the addition of exponential function signal or L1 signal usually adopted in existing adaptive event-triggered mechanisms. Then, the obtained results are extended from the strongly connected graph to the directed communication graph only containing a spanning tree. Finally, numerical simulation results are conducted to demonstrate the effectiveness of the proposed mechanism.

Adaptive Switched Control for Connected Vehicle Platoon With Unknown Input Delays.

A connected vehicle platoon with unknown input delays is studied in this article. The control objective is to stabilize the connected vehicles, ensuring all vehicles are traveling at the same speed while maintaining a safety spacing. A decentralized control law using only onboard sensors is designed for the connected vehicle platoon. A novel switching-type delay-adaptive predictor is proposed to estimate the unknown input delays. By using the estimated unknown input delays, the control law can guarantee the stability of the successive vehicles. The platoon control adopts a one-vehicle look-ahead topology structure and a constant time headway (CTH) policy, which makes the desired spacing between vehicles vary with time. In this framework, the stability of the connected vehicles can be derived through the analysis of each pair of two successive vehicles in the platoon. Finally, an example is presented to illustrate the applicability of the obtained results.

Unsupervised Monocular Depth Estimation With Channel and Spatial Attention.

Understanding 3-D scene geometry from videos is a fundamental topic in visual perception. In this article, we propose an unsupervised monocular depth and camera motion estimation framework using unlabeled monocular videos to overcome the limitation of acquiring per-pixel ground-truth depth at scale. The photometric loss couples the depth network and pose network together and is essential to the unsupervised method, which is based on warping nearby views to target using the estimated depth and pose. We introduce the channelwise attention mechanism to dig into the relationship between channels and introduce the spatialwise attention mechanism to utilize the inner-spatial relationship of features. Both of them applied in depth networks can better activate the feature information between different convolutional layers and extract more discriminative features. In addition, we apply the Sobel boundary to our edge-aware smoothness for more reasonable accuracy, and clearer boundaries and structures. All of these help to close the gap with fully supervised methods and show high-quality state-of-the-art results on the KITTI benchmark and great generalization performance on the Make3D dataset.

Distributed Localization for Multi-Agent Systems With Random Noise Based on Iterative Learning.

This article is concerned with the real-time localization problem for the dynamic multi-agent systems with measurement and communication noises under directed graphs. The barycentric coordinates are introduced to describe the relative position between agents. A novel robust distributed localization estimation algorithm based on iterative learning is proposed. The relative-distance unbiased estimator constructed from the historical iterative information is used to suppress the measurement noise. The designed stochastic approximation method with two iterative-varying gains is used to inhibit the communication noise. Under the zero-mean and independent distributed conditions on the measurement and communication noises, the asymptotic convergence of the proposed methods is derived. The numerical simulation and the QBot-2e robot experiment are conducted to test and verify the effectiveness and the practicability of the proposed methods.

Distributed Event-Triggered Control for Cooperative Output Regulation of Multiagent Systems With an Online Estimation Algorithm.

In this article, the cooperative output regulation problem of heterogeneous multiagent systems has been investigated. It is assumed that only a few agents could know the system matrix of the exosystem and no agent knows the topological information. Under these conditions, a novel distributed online algorithm is proposed to estimate the information relevant to the topology. Based on this algorithm, a distributed event-triggered adaptive observer is designed such that each agent can observe the exosystem. It is theoretically shown that the proposed distributed controller will make the multiagent system achieve cooperative output regulation asymptotically. Finally, a simulation is presented to show the effectiveness of the result.

Passivity-Based Output Synchronization With Switching Graphs and Transmission Delays.

In this article, the output synchronization problem of passive multiagent systems (MASs) with transmission delays and switching graphs is addressed by a novel logic-based distributed switching mechanism. Our result shows that synchronization is reached for arbitrarily large and bounded constant, time varying, or distributed delays, which, compared with the existing results for passive MASs, has an obvious advantage. This delay robustness holds under the very weak connectivity assumptions on the underlying graph, that is, as long as the graph is uniformly jointly strongly connected and switches with a dwell time. The proposed algorithm is applied to the position synchronization problem of multiple robotic manipulators to show its applicability.

Distributed Event-Triggered Consensus of General Linear Multiagent Systems Under Directed Graphs.

This article investigates the consensus problem of general linear multiagent systems under directed communication graphs with event-triggered mechanisms. First, a novel distributed static event-triggered mechanism with a state-dependent threshold is proposed to solve the consensus problem, both with a positive lower bound on the average time interval of the communication among agents and updates of controllers. Thus, the Zeno behavior is excluded for communication among agents and controller updates. Next, to further reduce the frequencies of communication among agents and updates of controllers, a distributed dynamic event-triggered mechanism is introduced. By applying the static and dynamic mechanisms, the problem can be addressed with the reduced use of system resources compared with that in most existing control algorithms. Finally, numerical simulations are presented to verify the effectiveness of the results.

Data-Based Optimal Consensus Control for Multiagent Systems With Policy Gradient Reinforcement Learning.

This article investigates the optimally distributed consensus control problem for discrete-time multiagent systems with completely unknown dynamics and computational ability differences. The problem can be viewed as solving nonzero-sum games with distributed reinforcement learning (RL), and each agent is a player in these games. First, to guarantee the real-time performance of learning algorithms, a data-based distributed control algorithm is proposed for multiagent systems using offline system interaction data sets. By utilizing the interactive data produced during the run of a real-time system, the proposed algorithm improves system performance based on distributed policy gradient RL. The convergence and stability are guaranteed based on functional analysis and the Lyapunov method. Second, to address asynchronous learning caused by computational ability differences in multiagent systems, the proposed algorithm is extended to an asynchronous version in which executing policy improvement or not of each agent is independent of its neighbors. Furthermore, an actor-critic structure, which contains two neural networks, is developed to implement the proposed algorithm in synchronous and asynchronous cases. Based on the method of weighted residuals, the convergence and optimality of the neural networks are guaranteed by proving the approximation errors converge to zero. Finally, simulations are conducted to show the effectiveness of the proposed algorithm.

Active Full-Vehicle Suspension Control via Cloud-Aided Adaptive Backstepping Approach.

This paper is concerned with the adaptive backstepping control problem for a cloud-aided nonlinear active full-vehicle suspension system. A novel model for a nonlinear active suspension system is established, in which uncertain parameters, unknown friction forces, nonlinear springs and dampers, and performance requirements are considered simultaneously. In order to deal with the nonlinear characteristics, a backstepping control strategy is developed. Meanwhile, an adaptive control strategy is proposed to handle the uncertain parameters and unknown friction forces. In the cloud-aided vehicle suspension system framework, the adaptive backstepping controller is updated in a remote cloud based on the cloud storing information, such as road information, vehicle suspension information, and reference trajectories. Finally, simulation results for a full vehicle with 7-degree of freedom model are provided to demonstrate the effectiveness of the proposed control scheme, and it is shown that the addressed controller can improve the performances more than 80% compared with passive vehicle suspension systems.

Maneuvering Target Tracking With Event-Based Mixture Kalman Filter in Mobile Sensor Networks.

In this paper, the distributed remote state estimation problem for conditional dynamic linear systems in mobile sensor networks with an event-triggered mechanism is investigated. The distributed mixture Kalman filtering method is proposed to track the state of the maneuvering target, which uses particle filtering to estimate the nonlinear variables and apply Kalman filtering to estimate the linear variables. An event-based distributed filtering scheme is designed, which is an energy-efficient way to transmit data between sensors and estimators. In addition, by using the mutual information theory, an optimal control problem is formed to control the position of sensors so that the target tracking process can be achieved quickly. Finally, a simulation example about the maneuvering target tracking is provided to corroborate the effectiveness of the filtering method and the control performance for sensors.

Adaptive Consensus-Based Distributed Target Tracking With Dynamic Cluster in Sensor Networks.

This paper is concerned with the target tracking problem over a filtering network with dynamic cluster and data fusion. A novel distributed consensus-based adaptive Kalman estimation is developed to track a linear moving target. Both optimal filtering gain and average disagreement of the estimates are considered in the filter design. In order to estimate the states of the target more precisely, an optimal Kalman gain is obtained by minimizing the mean-squared estimation error. An adaptive consensus factor is employed to adjust the optimal gain as well as to acquire a better filtering performance. In the filter's information exchange, dynamic cluster selection and two-stage hierarchical fusion structure are employed to get more accurate estimation. At the first stage, every sensor collects information from its neighbors and runs the Kalman estimation algorithm to obtain a local estimate of system states. At the second stage, each local sensor sends its estimate to the cluster head to get a fused estimation. Finally, an illustrative example is presented to validate the effectiveness of the proposed scheme.

Adaptive Event-Triggered Transmission Scheme and H∞ Filtering Co-Design Over a Filtering Network With Switching Topology.

This paper addresses the distributed adaptive event-triggered H∞ filtering problem for a class of sector-bounded nonlinear system over a filtering network with time-varying and switching topology. Both topology switching and adaptive event-triggered mechanisms (AETMs) between filters are simultaneously considered in the filtering network design. The communication topology evolves over time, which is assumed to be subject to a nonhomogeneous Markov chain. In consideration of the limited network bandwidth, AETMs have been used in the information transmission from the sensor to the filter as well as the information exchange among filters. The proposed AETM is characterized by introducing the dynamic threshold parameter, which provides benefits in data scheduling. Moreover, the gain of the correction term in the adaptive rule varies directly with the estimation error and inversely with the transmission error. The switching filtering network is modeled by a Markov jump nonlinear system. The stochastic Markov stability theory and linear matrix inequality techniques are exploited to establish the existence of the filtering network and further derive the filter parameters. A co-design algorithm for determining H∞ filters and the event parameters is developed. Finally, some simulation results on a continuous stirred tank reactor and a numerical example are presented to show the applicability of the obtained results.

Reinforcement Learning-Based Satellite Attitude Stabilization Method for Non-Cooperative Target Capturing.

When a satellite performs complex tasks such as discarding a payload or capturing a non-cooperative target, it will encounter sudden changes in the attitude and mass parameters, causing unstable flying and rolling of the satellite. In such circumstances, the change of the movement and mass characteristics are unpredictable. Thus, the traditional attitude control methods are unable to stabilize the satellite since they are dependent on the mass parameters of the controlled object. In this paper, we proposed a reinforcement learning method to re-stabilize the attitude of a satellite under such circumstances. Specifically, we discretize the continuous control torque, and build a neural network model that can output the discretized control torque to control the satellite. A dynamics simulation environment of the satellite is built, and the deep Q Network algorithm is then performed to train the neural network in this simulation environment. The reward of the training is the stabilization of the satellite. Simulation experiments illustrate that, with the iteration of training progresses, the neural network model gradually learned to re-stabilize the attitude of a satellite after unknown disturbance. As a contrast, the traditional PD (Proportion Differential) controller was unable to re-stabilize the satellite due to its dependence on the mass parameters. The proposed method adopts self-learning to control satellite attitudes, shows considerable intelligence and certain universality, and has a strong application potential for future intelligent control of satellites performing complex space tasks.

Imbalance of Peripheral Th17 and Regulatory T Cells in Children with Allergic Rhinitis and Bronchial Asthma.

The purpose of the present study is to investigate the prevalence of Th17 and regulatory T (Treg) cells in children with allergic rhinitis (AR) accompanying with bronchial asthma (BA). 24 children with AR, 22 children with BA, 18 children with AR accompanying with BA, and 20 healthy controls were recruited. The prevalence of peripheral blood Th17 and Treg cells were determined by flow cytometry. mRNA expression of retinoid-acid receptor-related orphan receptor (ROR)-γt and forkhead box P3 (Foxp3) were determined by realtime polymerase chain reaction. Cytokine expressions in plasma were determined by enzyme linked immunosorbent assay. The frequency of Th17 cells, ROR-γt mRNA expression, and the plasma levels of IL-17 were significantly higher, while Treg cells and Transforming growth factor (TGF)-ß1 were significantly lower in children with AR accompanying with BA compared with those in children with AR or BA alone or control subjects. In children with allergic airway disease, total IgE levels were positively correlated to the frequency of Th17 cells (r=0.607, p<0.01), plasma IL-17 levels, and negatively correlated to the frequency of Treg cells (r=-0.429, p<0.01) and TGF-ß1 levels (r=-0.224, p<0.01). While Forced expiratory volume in one second (FEV1) (% predicted) was negatively correlated to the frequency of Th17 cells (r=-0.602, p<0.01), plasma IL-17 levels (r=-0.577, p<0.01), and positively correlated to the frequency of Treg cells r=0.504, p<0.01) and TGF-ß1 levels (r=0.231, p<0.05). Our results demonstrate that the imbalance of peripheral Th17/Treg cells plays an important role in the pathogenesis of AR accompanying with BA.