ATM-R: An Adaptive Tradeoff Model With Reference Points for Constrained Multiobjective Evolutionary Optimization.

The goal of constrained multiobjective evolutionary optimization is to obtain a set of well-converged and well-distributed feasible solutions. To achieve this goal, a delicate tradeoff must be struck among feasibility, diversity, and convergence. However, balancing these three elements simultaneously through a single tradeoff model is nontrivial, mainly because the significance of each element varies in different evolutionary phases. As an alternative approach, we adapt distinct tradeoff models in various phases and introduce a novel algorithm named adaptive tradeoff model with reference points (ATM-R). In the infeasible phase, ATM-R takes the tradeoff between diversity and feasibility into account, aiming to move the population toward feasible regions from diverse search directions. In the semi-feasible phase, ATM-R promotes the transition from "the tradeoff between feasibility and diversity" to "the tradeoff between diversity and convergence." This transition is instrumental in discovering an adequate number of feasible regions and accelerating the search for feasible Pareto optima in succession. In the feasible phase, ATM-R places an emphasis on balancing diversity and convergence to obtain a set of feasible solutions that are both well-converged and well-distributed. It is worth noting that the merits of reference points are leveraged in ATM-R to accomplish these tradeoff models. Also, in ATM-R, a multiphase mating selection strategy is developed to generate promising solutions beneficial to different evolutionary phases. Systemic experiments on a diverse set of benchmark test functions and real-world problems demonstrate that ATM-R is effective. When compared to eight state-of-the-art constrained multiobjective optimization evolutionary algorithms, ATM-R consistently demonstrates its competitive performance.

Human-in-the-Loop Reinforcement Learning in Continuous-Action Space.

Human-in-the-loop for reinforcement learning (RL) is usually employed to overcome the challenge of sample inefficiency, in which the human expert provides advice for the agent when necessary. The current human-in-the-loop RL (HRL) results mainly focus on discrete action space. In this article, we propose a Q value-dependent policy (QDP)-based HRL (QDP-HRL) algorithm for continuous action space. Considering the cognitive costs of human monitoring, the human expert only selectively gives advice in the early stage of agent learning, where the agent implements human-advised action instead. The QDP framework is adapted to the twin delayed deep deterministic policy gradient algorithm (TD3) in this article for the convenience of comparison with the state-of-the-art TD3. Specifically, the human expert in the QDP-HRL considers giving advice in the case that the difference between the twin Q -networks' output exceeds the maximum difference in the current queue. Moreover, to guide the update of the critic network, the advantage loss function is developed using expert experience and agent policy, which provides the learning direction for the QDP-HRL algorithm to some extent. To verify the effectiveness of QDP-HRL, the experiments are conducted on several continuous action space tasks in the OpenAI gym environment, and the results demonstrate that QDP-HRL greatly improves learning speed and performance.

Spatial Decomposition-Based Fault Detection Framework for Parabolic-Distributed Parameter Processes.

Fault detection for distributed parameter processes (heat processes, fluid processes, etc.) is vital for safe and efficient operation. On one hand, the existing data-driven methods neglect the evolution dynamics of the processes and cannot guarantee that they work for highly dynamic or transient processes; on the other hand, model-based methods reported so far are mostly based on the backstepping technique, which does not possess enough redundancy for fault detection since only the boundary measurement is considered. Motivated by these considerations, we intend to investigate the robust fault detection problem for distributed parameter processes in a model-based perspective covering both boundary and in-domain measurement cases. A real-time fault detection filter (FDF) is presented, which gets rid of a large amount of data collection and offline training procedures. Rigorous theoretic analysis is presented for guiding the parameters selection and threshold computation. A time-varying threshold is designed such that the false alarm in the transient stage can be avoided. Successful application results on a hot strip mill cooling system demonstrate the potential for real industrial applications.

Handling Constrained Multiobjective Optimization Problems via Bidirectional Coevolution.

Constrained multiobjective optimization problems (CMOPs) involve both conflicting objective functions and various constraints. Due to the presence of constraints, CMOPs' Pareto-optimal solutions are very likely lying on constraint boundaries. The experience from the constrained single-objective optimization has shown that to quickly obtain such an optimal solution, the search should surround the boundary of the feasible region from both the feasible and infeasible sides. In this article, we extend this idea to cope with CMOPs and, accordingly, we propose a novel constrained multiobjective evolutionary algorithm with bidirectional coevolution, called BiCo. BiCo maintains two populations, that is: 1) the main population and 2) the archive population. To update the main population, the constraint-domination principle is equipped with an NSGA-II variant to move the population into the feasible region and then to guide the population toward the Pareto front (PF) from the feasible side of the search space. While for updating the archive population, a nondominated sorting procedure and an angle-based selection scheme are conducted in sequence to drive the population toward the PF within the infeasible region while maintaining good diversity. As a result, BiCo can get close to the PF from two complementary directions. In addition, to coordinate the interaction between the main and archive populations, in BiCo, a restricted mating selection mechanism is developed to choose appropriate mating parents. Comprehensive experiments have been conducted on three sets of CMOP benchmark functions and six real-world CMOPs. The experimental results suggest that BiCo can obtain quite competitive performance in comparison to eight state-of-the-art-constrained multiobjective evolutionary optimizers.

Incorporating Objective Function Information Into the Feasibility Rule for Constrained Evolutionary Optimization.

When solving constrained optimization problems by evolutionary algorithms, an important issue is how to balance constraints and objective function. This paper presents a new method to address the above issue. In our method, after generating an offspring for each parent in the population by making use of differential evolution (DE), the well-known feasibility rule is used to compare the offspring and its parent. Since the feasibility rule prefers constraints to objective function, the objective function information has been exploited as follows: if the offspring cannot survive into the next generation and if the objective function value of the offspring is better than that of the parent, then the offspring is stored into a predefined archive. Subsequently, the individuals in the archive are used to replace some individuals in the population according to a replacement mechanism. Moreover, a mutation strategy is proposed to help the population jump out of a local optimum in the infeasible region. Note that, in the replacement mechanism and the mutation strategy, the comparison of individuals is based on objective function. In addition, the information of objective function has also been utilized to generate offspring in DE. By the above processes, this paper achieves an effective balance between constraints and objective function in constrained evolutionary optimization. The performance of our method has been tested on two sets of benchmark test functions, namely, 24 test functions at IEEE CEC2006 and 18 test functions with 10-D and 30-D at IEEE CEC2010. The experimental results have demonstrated that our method shows better or at least competitive performance against other state-of-the-art methods. Furthermore, the advantage of our method increases with the increase of the number of decision variables.