Deep Reinforcement Learning for Autonomous Driving with an Auxiliary Actor Discriminator.

In the research of robot systems, path planning and obstacle avoidance are important research directions, especially in unknown dynamic environments where flexibility and rapid decision makings are required. In this paper, a state attention network (SAN) was developed to extract features to represent the interaction between an intelligent robot and its obstacles. An auxiliary actor discriminator (AAD) was developed to calculate the probability of a collision. Goal-directed and gap-based navigation strategies were proposed to guide robotic exploration. The proposed policy was trained through simulated scenarios and updated by the Soft Actor-Critic (SAC) algorithm. The robot executed the action depending on the AAD output. Heuristic knowledge (HK) was developed to prevent blind exploration of the robot. Compared to other methods, adopting our approach in robot systems can help robots converge towards an optimal action strategy. Furthermore, it enables them to explore paths in unknown environments with fewer moving steps (showing a decrease of 33.9%) and achieve higher average rewards (showning an increase of 29.15%).

Effects of short- and long-term plant functional group loss on alpine meadow community structure and soil nutrients.

The rapid loss of global biodiversity can greatly affect the normal functioning of ecosystems. However, how biodiversity losses affect plant community structure and soil nutrients is unclear. We conducted a field experiment to examine the short- and long-term effects of removing plant functional groups (Gramineae, Cyperaceae, legumes, and forbs) on the interrelationships among the species diversity, productivity, community structure, and soil nutrients in an alpine meadow ecosystem at Menyuan County, Qinghai Province. The variations in the species richness, above- and belowground biomass of the community gradually decreased over time. Species richness and productivity were positively correlated, and this correlation tended to be increasingly significant over time. Removal of the Cyperaceae, legumes, and other forbs resulted in fewer Gramineae species in the community. Soil total nitrogen, phosphorus, organic matter, and moisture contents increased significantly in the legume removal treatment. The removal of other forbs led to the lowest negative cohesion values, suggesting that this community may have difficulty recovering its previous equilibrium state within a short time. The effects of species removal on the ecosystem were likely influenced by the species structure and composition within the community. Changes in the number of Gramineae species indicated that they were more sensitive and less resistant to plant functional group removal. Legume removal may also indirectly cause distinct community responses through starvation and compensation effects. In summary, species loss at the community level led to extensive species niche shifts, which caused community resource redistribution and significant changes in community structure.

Accelerating reinforcement learning with case-based model-assisted experience augmentation for process control.

In the context of intelligent manufacturing in the process industry, traditional model-based optimization control methods cannot adapt to the situation of drastic changes in working conditions or operating modes. Reinforcement learning (RL) directly achieves the control objective by interacting with the environment, and has significant advantages in the presence of uncertainty since it does not require an explicit model of the operating plant. However, most RL algorithms fail to retain transfer learning capabilities in the presence of mode variation, which becomes a practical obstacle to industrial process control applications. To address these issues, we design a framework that uses local data augmentation to improve the training efficiency and transfer learning (adaptability) performance. Therefore, this paper proposes a novel RL control algorithm, CBR-MA-DDPG, organically integrating case-based reasoning (CBR), model-assisted (MA) experience augmentation, and deep deterministic policy gradient (DDPG). When the operating mode changes, CBR-MA-DDPG can quickly adapt to the varying environment and achieve the desired control performance within several training episodes. Experimental analyses on a continuous stirred tank reactor (CSTR) and an organic Rankine cycle (ORC) demonstrate the superiority of the proposed method in terms of both adaptability and control performance/robustness. The results show that the control performance of the CBR-MA-DDPG agent outperforms the conventional PI and MPC control schemes, and that it has higher training efficiency than the state-of-the-art DDPG, TD3, and PPO algorithms in transfer learning scenarios with mode shift situations.

Optimal Control for Constrained Discrete-Time Nonlinear Systems Based on Safe Reinforcement Learning.

The state and input constraints of nonlinear systems could greatly impede the realization of their optimal control when using reinforcement learning (RL)-based approaches since the commonly used quadratic utility functions cannot meet the requirements of solving constrained optimization problems. This article develops a novel optimal control approach for constrained discrete-time (DT) nonlinear systems based on safe RL. Specifically, a barrier function (BF) is introduced and incorporated with the value function to help transform a constrained optimization problem into an unconstrained one. Meanwhile, the minimum of such an optimization problem can be guaranteed to occur at the origin. Then a constrained policy iteration (PI) algorithm is developed to realize the optimal control of the nonlinear system and to enable the state and input constraints to be satisfied. The constrained optimal control policy and its corresponding value function are derived through the implementation of two neural networks (NNs). Performance analysis shows that the proposed control approach still retains the convergence and optimality properties of the traditional PI algorithm. Simulation results of three examples reveal its effectiveness.

SVD-Based Robust Distributed MPC for Tracking Systems Coupled in Dynamics With Global Constraints.

This article presents a novel singular value decomposition (SVD)-based robust distributed model predictive control (SVD-RDMPC) strategy for linear systems with additive uncertainties. The system is globally constrained and consists of multiple interrelated subsystems with bounded disturbances, each of whom has local constraints on states and inputs. First, we integrate the steady-state target optimizer into the MPC problem through the offset cost function to formulate a modified single optimization problem for tracking changing targets from real-time optimization. Then, the concept of constraint tightening is utilized to enhance the robustness and ensure robust constraint satisfaction in the presence of interferences. On this basis, the SVD method is introduced to decompose the new optimization problem into several independent subsystems on the orthogonal projection space, and a distributed dual gradient algorithm with convergence proved is implemented to obtain the control of each nominal subsystem. The recursive feasibility is then ensured and the tracking ability of the strategy is analyzed. It is verified that for a target, the system can be steered to a neighborhood of the closest possible steady setpoint. At last, the effectiveness of the raised SVD-RDMPC strategy is established in two simulations on building temperature control and load frequency control.

Automated rock mass condition assessment during TBM tunnel excavation using deep learning.

Rock mass condition assessment during tunnel excavation is a critical step for the intelligent control of tunnel boring machine (TBM). To address this and achieve automatic detection, a visual assessment system is installed to the TBM and a lager in-situ rock mass image dataset is collected from the water conveyance channel project. The rock mass condition assessment task is transformed into a fine-grain classification task. To fulfill the task, a self-convolution based attention fusion network (SAFN) is designed in this paper. The core of our method is the discovery and fusion of the object attention map within a deep neural network. The network consists of two novel modules, the self-convolution based attention extractor (SAE) module and the self-convolution based attention pooling algorithm (SAP) module. The former is designed to detect the intact rock regions generating the attention map, and the latter is designed to improve the performance of classifier by fusing the attention map that focuses on the intact rock regions. The results of SAFN are evaluated from aspects of interpretability, ablation, accuracy and cross-validation, and it outperforms state-of-the-art models in the rock mass assessment dataset. Furthermore, the dynamic filed test show that our assessment system based on the SAFN model is accurate and efficient for automated classification of rock mass.

Event-Triggered Constrained Optimal Control for Organic Rankine Cycle Systems via Safe Reinforcement Learning.

The organic Rankine cycle (ORC) is an effective application for converting low-grade heat sources into power and is crucial for environmentally friendly production and energy recovery. However, the inherent complexity of the mechanism, its strong and unidentified nonlinearity, and the presence of control constraints severely impair the design of its optimal controller. To solve these issues, this study provides a novel event-triggered (ET) constrained optimal control approach for the ORC systems based on a safe reinforcement learning technique to find the optimal control law. Instead of employing the usual non-quadratic integral form to solve the control-limited optimal control problems, a constraint handling strategy based on a relaxed weighted barrier function (BF) technique is proposed. By adding the BF terms to the original value function, a modified value iteration algorithm is developed to make the control input solutions that tend to violate the constraints be pushed back and maintained in their safe sets. In addition, the ET mechanism proposed in this article is critically required for the ORC systems, and it can significantly reduce the computational load. The combination of these two techniques allows the ORC systems to achieve set-point tracking control and satisfy the control restrictions. The proposed approach is conducted based on a heuristic dynamic programming framework with three neural networks (NNs) involved. The safety and convergence of the proposed approach and the stability of the closed-loop system are analyzed. Simulation results and comparisons are presented to demonstrate its effectiveness.

Quality-Relevant Process Monitoring with Concurrent Locality-Preserving Dynamic Latent Variable Method.

A concurrent locality-preserving dynamic latent variable (CLDLV) method is proposed to extract the correlation between process variables and quality variables for quality-related dynamic process monitoring. Given that dynamic process data can easily be contaminated by noise and outliers and conventional dynamic latent variable models lack robustness, a low-rank autoregressive model is developed to deal with autocorrelation and cross-correlation properties among the data. Then neighborhood structure information is integrated into the partial least squares model, which can better reveal the essential structure of the data. The final concurrent projection of the latent structures is employed to monitor output-related faults and input-related process faults that affect quality. The Tennessee Eastman process and hot strip mill process are used to demonstrate the effectiveness of CLDLV-based detection and diagnostic methods.

Expression of LncRNA KCNQ1Ot1 in diabetic nephropathy and its correlation with MEK/ERK signaling pathway.

OBJECTIVE: To explore the expression of LncRNA KCNQ1OT1 in diabetic nephropathy (DN), and its correlation with MEK/ERK signaling pathway. METHODS: 148 patients with type 2 diabetes in our hospital were selected as research subjects, including 83 patients with simple type 2 diabetes (T2D group) and 65 patients with type 2 diabetes with DN (DN group). Another 50 non-diabetic patients were enrolled as the control group. The expressions of LncRNA KCNQ1OT1 and MEK/ERK signaling pathway related molecules in peripheral blood mononuclear cells (PBMCs) of the three groups of subjects were detected and their correlations were analyzed. In addition, 30 Wistar rats were divided into a control group, diabetes group and DN model group, and the expression of LncRNA KCNQ1OT1 and MEK/ERK signal pathway-related molecules in kidney tissue of the three groups was detected and compared. RESULTS: The relative expression of LncRNA KCNQ1OT1, MEK-5 and ERK2 in the control group was lower than that of the T2D group and DN group (P<0.05), and the relative expression of LncRNA KCNQ1OT1 in T2D group was lower than that of DN group (P<0.05). The expression of LncRNA KCNQ1OT1 was positively-correlated with MEK-5 and ERK2 (P<0.05). The relative expression of LncRNA KCNQ1OT1, MEK-5, and ERK2 in renal tissues of the DN group was higher than those in the control group and diabetes group (P<0.05). CONCLUSION: The expression of LncRNA KCNQ1OT1 in PBMCs of DN patients is abnormally increased, and may be a biomarker for the diagnosis and treatment of the disease. In addition, an abnormal increase of LncRNA KCNQ1OT1 is associated with the activation of the MEK/ERK signaling pathway.

Biodiversity and Ecosystem Function under Simulated Gradient Warming and Grazing.

Biodiversity and ecosystem functions and their relationship with environmental response constitute a major topic of ecological research. However, the changes in and impact mechanisms of multi-dimensional biodiversity and ecosystem functions in continuously changing environmental gradients and anthropogenic activities remain poorly understood. Here, we analyze the effects of multi-gradient warming and grazing on relationships between the biodiversity of plant and soil microbial with productivity/community stability through a field experiment simulating multi-gradient warming and grazing in alpine grasslands on the Tibetan Plateau. We show the following results: (i) Plant biodiversity, soil microbial diversity and community productivity in alpine grasslands show fluctuating trends with temperature gradients, and a temperature increase below approximately 1 °C is beneficial to alpine grasslands; moderate grazing only increases the fungal diversity of the soil surface layer. (ii) The warming shifted plant biomass underground in alpine grasslands to obtain more water in response to the decrease in soil moisture caused by the temperature rise. Community stability was not affected by warming or grazing. (iii) Community stability was not significantly correlated with productivity, and environmental factors, rather than biodiversity, influenced community stability and productivity.

Effects of Pedicularis kansuensis Expansion on Plant Community Characteristics and Soil Nutrients in an Alpine Grassland.

Pedicularis kansuensis is an indicator species of grassland degradation. Its population expansion dramatically impacts the production and service function of the grassland ecosystem, but the effects and mechanisms of the expansion are still unclear. In order to understand the ecological effects of P. kansuensis, three P. kansuensis patches of different densities were selected in an alpine grassland, and species diversity indexes, biomasses, soil physicochemical properties, and the mechanism among them were analyzed. The results showed that P. kansuensis expansion increased the richness index, the Shannon−Wiener index significantly, and the aboveground biomass ratio (ABR) of the Weed group (p < 0.05), but reduced the total biomass of the community and the ABR of the Gramineae and Cyperaceae decreased insignificantly (p > 0.05); soil moisture, soil AOC, and NO3−·N decreased significantly (p < 0.05), while soil pH and total soil nutrients did not change significantly, and available phosphorus (AP) decreased at first and then increased (p < 0.05). The structural equation model (SEM) showed that P. kansuensis expansion had a significant positive effect on the community richness index, and a significant negative effect followed on the soil AOC from the increase of the index; the increase of pH had a significant negative effect on the soil AOC, NO3−·N, and AP. It indicated that P. kansuensis expansion resulted in the increase of species richness, the ABR of the Weed group, and the community's water demand, which promoted the over-utilization of soil available nutrients in turn, and finally caused the decline of soil quality. This study elucidated a possible mechanism of poisonous weeds expansion, and provided a scientific and theoretical basis for grassland management.

An improved mixture robust probabilistic linear discriminant analyzer for fault classification.

This article introduces a novel fault classification method based on the mixture robust probabilistic linear discriminant analysis (MRPLDA). Unlike conventional probabilistic models like probabilistic principal component analysis (PPCA), probabilistic linear discriminant analysis (PLDA) introduces two sets of latent variables to represent the within-class and between-class information, resulting in an enhanced classification capability. In order to deal with outliers and non-Gaussian distributed variables commonly encountered in industrial processes, a mixture of robust PLDA model is considered by imposing the Student's t-priors on the noise and hidden variables of the PLDA model. Based on the model, a variational Bayesian expectation-maximization algorithm is developed for parameter estimation. In order to determine the state/class of a test sample, this article proposes a new state inference method by considering the joint probability between the test and training samples. The state inference method consists of a probability approximation, an evidence inference, and a voting based decision stage. The performance of the proposed fault classification method is illustrated by a numerical example and an application study to the Tennessee Eastman (TE) process.

Quantized Feedback Control of Fuzzy Markov Jump Systems.

This paper addresses the problem of quantized feedback control of nonlinear Markov jump systems (MJSs). The nonlinear plant is represented by a class of fuzzy MJSs with time-varying delay based on a Takagi-Sugeno fuzzy model. The quantized signal is utilized for control purpose and the sector bound approach is exploited to deal with quantization errors. By constructing a Lyapunov function which depends both on mode information and fuzzy basis functions, the reciprocally convex approach is used to derive the criterion which is able to ensure the stochastic stability with a predefined l2-l∞ performance of the resulting closed-loop system. The design of the quantized feedback controller is then converted to a convex optimization problem, which can be handled through the linear matrix inequality technique. Finally, a simulation example is presented to verify the effectiveness and practicability of the proposed new design techniques.

Effects of liraglutide on lipolysis and the AC3/PKA/HSL pathway.

BACKGROUND: Liraglutide reduces blood glucose, body weight and blood lipid levels. Hormone-sensitive lipase (HSL) is a key enzyme in lipolysis. Evidence from our and other studies have demonstrated that adenylate cyclase 3 (AC3) is associated with obesity and can be upregulated by liraglutide in obese mice. In the present study, we investigated whether hepatic HSL activity is regulated by liraglutide and characterized the effect of liraglutide in the AC3/protein kinase A (PKA)/HSL signalling pathway. METHODS: Obese mice or their lean littermates were treated with liraglutide or saline for 8 weeks. Serum was collected for the measurement of insulin and lipids. We investigated hepatic AC3, HSL and phosphorylated HSL Ser-660 (p-HSL(S660)) protein expression levels andAC3 and HSL mRNA expression levels and cyclic adenosine monophosphate (cAMP), PKA activity in liver tissue. RESULTS: Liraglutide treatment decreased triglycerides (TGs) and free fatty acids (FFAs), increased glycerol, and upregulated hepatic AC3 and p-HSL(s660) levels and cAMP and PKA activities. CONCLUSION: The results suggest that liraglutide can upregulates AC3/PKA/HSL pathway and may promotes lipolysis.

Quantized Control of Markov Jump Nonlinear Systems Based on Fuzzy Hidden Markov Model.

This paper considers the problem of asynchronous guaranteed cost control (GCC) for nonlinear Markov jump systems with stochastic quantization. Hidden Markov model is used to describe the nonsynchronous controller and the random quantization phenomenon. Based on Takagi-Sugeno fuzzy technique and Lyapunov function approach, a sufficient condition is obtained, which can not only ensure the asymptotic stability of the closed-loop system and existence of the desired controller, but also can yield the minimal upper bound of GCC performance. Finally, two examples are provided to demonstrate the correctness and reliability of our developed approaches.

Asynchronous and Resilient Filtering for Markovian Jump Neural Networks Subject to Extended Dissipativity.

The problem of asynchronous and resilient filtering for discrete-time Markov jump neural networks subject to extended dissipativity is investigated in this paper. The modes of the designed resilient filter are assumed to run asynchronously with the modes of original Markov jump neural networks, which accord well with practical applications and are described through a hidden Markov model. Due to the fluctuation of the filter parameters, a resilient filter taking into account parameter uncertainty is adopted. Being different from the norm-bound type of uncertainty which has been studied in a considerable number of the existing literatures, the interval type of uncertainty is introduced so as to describe uncertain phenomenon more accurately. By means of convex optimal method, the gains of filter are derived to guarantee the stochastic stability and extended dissipativity of the filtering error system under the wave of the filter parameters. Considering the limited computing power of MATLAB solver, a relatively simple simulation is exploited to verify the effectiveness and merits of the theoretical findings where the relationships among optimal performance index, uncertain parameter σ , and asynchronous rate are revealed.

Hidden-Markov-Model-Based Asynchronous Filter Design of Nonlinear Markov Jump Systems in Continuous-Time Domain.

This paper addresses the dissipative asynchronous filtering problem for a class of Takagi-Sugeno fuzzy Markov jump systems in the continuous-time domain. The hidden Markov model is applied to describe the asynchronous situation between the designed filter and the original system. Based on the stochastic Lyapunov function, a sufficient condition is developed to guarantee the stochastic stability of the filtering error systems with a given dissipative performance. Two different methods for the existence of desired filter are established. Due to the Finsler's lemma, the second approach has fewer variables to decide and brings less conservatism than the first one. Finally, an example is provided to demonstrate the correctness and advantage of the proposed approaches.

Asynchronous Dissipative Control for Fuzzy Markov Jump Systems.

The problem of asynchronous dissipative control is investigated for Takagi-Sugeno fuzzy systems with Markov jump in this paper. Hidden Markov model is introduced to represent the nonsynchronization between the designed controller and the original system. By the fuzzy-basis-dependent and mode-dependent Lyapunov function, a sufficient condition is achieved such that the resulting closed-loop system is stochastically stable with a strictly ( , , )- -dissipative performance. The controller parameter is derived by applying MATLAB to solve a set of linear matrix inequalities. Finally, we present two examples to confirm the validity and correctness of our developed approach.

H∞ Output Consensus for Markov Jump Multiagent Systems With Uncertainties.

This paper investigates the H∞ output consensus problem for multiagent systems with Markov jump and external disturbance in both continuous-time and discrete-time domains. The communication network is directed and fixed with uncertainties. Based on the hidden Markov model, an output feedback controller is constructed. Then, the original system is transformed into a reduced-order system, which features the error dynamics. By using a Lyapunov function, sufficient conditions are developed to ensure that all agents can reach the consensus with the desired H∞ performance in the mean-square sense. Finally, simulation results are presented to illustrate the efficiency of the proposed approaches.

Synchronization of General Chaotic Neural Networks With Nonuniform Sampling and Packet Missing: A Switched System Approach.

This paper is concerned with the exponential synchronization issue of general chaotic neural networks subject to nonuniform sampling and control packet missing in the frame of the zero-input strategy. Based on this strategy, we make use of the switched system model to describe the synchronization error system. First, when the missing of control packet does not occur, an exponential stability criterion with less conservatism is established for the resultant synchronization error systems via a superior time-dependent Lyapunov functional and the convex optimization approach. The characteristics induced by nonuniform sampling can be used to the full because of the structure and property of the constructed Lyapunov functional, that is not necessary to be positive definite except sampling times. Then, a criterion is obtained to guarantee that the general chaotic neural networks are synchronous exponentially when the missing of control packet occurs by means of the average dwell-time technique. An explicit expression of the sampled-data static output feedback controller is also gained. Finally, the effectiveness of the proposed new design methods is shown via two examples.