A Hybrid MPC for Constrained Deep Reinforcement Learning applied for Planar Robotic Arm.

Recently, deep reinforcement learning techniques have achieved tangible results for learning high dimensional control tasks. Due to the trial and error interaction, between the autonomous agent and the environment, the learning phase is unconstrained and limited to the simulator. Such exploration has an additional drawback of consuming unnecessary samples at the beginning of the learning process. Model-based algorithms, on the other hand, handle this issue by learning the dynamics of the environment. However, model-free algorithms have a higher asymptotic performance than model-based ones. The main contribution of this paper is to construct a hybrid structured algorithm from model predictive control (MPC) and deep reinforcement learning (DRL) (MPC-DRL), that makes use of the benefits of both methods, to satisfy constraint conditions throughout the learning process. The validity of the proposed approach is demonstrated by learning a reachability task. The results show complete satisfaction for the constraint condition, represented by a static obstacle, with a smaller number of samples and higher performance compared to state-of-the-art model-free algorithms.

Implementation of different intelligent controllers for mitigating the AC corrosion of metallic pipelines considering all HVOHTLs operation conditions.

AC corrosion represents a chronic issue that is faced by the buried metallic pipelines that are neighboring by the high voltage overhead transmission lines (HVOHTLs). In this paper a potassium hydroxide polarization cell (KOH-PC) is employed on the pipeline to mitigate the generated voltage from the power lines' intrusion with the buried pipeline. This work further investigates an integrated system to exploit the discharged energy for compensating the cathodic protection disturbances where a portion of this energy may be converted to DC form and reapplied on the pipeline as a cathodic protection voltage. This paper also presents a comparative analysis of different KOH-PCs' models with different controllers to provide the guidelines for deciding which one of these techniques is more suitable for mitigating the induced voltage and improving the cathodic protection distribution during normal and fault operating conditions of transmission lines. The applied controllers are the artificial-neural-network (ANN), Fuzzy-logic-controller (FLC), and​ adaptive-neuro-fuzzy inference system (ANFIS). The proposed model's performance is implemented through modeling and simulating on MATLAB software with the experimental measurements from the comparative analysis. The ANFIS controller is more precise in differentiation with other controllers in compensating for the CP disturbances.

Deep learning for Koopman Operator Optimal Control.

Nonlinear dynamics are ubiquitous in complex systems. Their applications range from robotics to computational neuroscience. In this work, the Koopman framework for globally linearizing nonlinear dynamics is introduced. Under this framework, the nonlinear observable states are lifted into a higher dimensional, linear regime. The challenge is to identify functions that facilitate the coordinate transformation to this raised linear space. This point is tackled using deep learning, where nonlinear dynamics are learned in a model-free manner, i.e., the underlying dynamics are uncovered using data rather than the nonlinear state-space equations. The main contributions include an implementation of the Linearly Recurrent Encoder Network (LREN) that is faster than the existing implementation and is significantly faster than the state-of-the-art deep learning-based approach. Also, a novel architecture termed Deep Encoder with Initial State Parameterization (DENIS) is proposed. By deriving an energy-budget control performance evaluation method, we demonstrate that DENIS also outperforms LREN in control performance. It is also on-par with and sometimes better than the iterative linear quadratic regulator (iLQR), which requires access to the state-space equations. Extensive experiments are done on DENIS to validate its performance. Also, another novel architecture termed Double Encoder for Input Nonaffine systems (DEINA) is described. Additionally, DEINA's potential ability to outperform existing Koopman frameworks for controlling nonaffine input systems is shown. We attribute this to using an auxiliary network to nonlinearly transform the inputs, thereby lifting the strong linear constraints imposed by the traditional Koopman approximation approach. Koopman model predictive control (KMPC) is implemented to verify that our models can also be successfully controlled under this popular approach. Overall, we demonstrate the deep learning-based Koopman framework shows promise for optimally controlling nonlinear dynamics.

Modeling of the KOH-Polarization cells for mitigating the induced AC voltage in the metallic pipelines.

The problem of AC corrosion remains motivating for researchers because many factors influence the corrosion rate for buried pipelines due to the interference with overhead high voltage transmission lines (OHVTLs). Many researchers study the mechanisms of induced alternating current (AC) voltages, which are summarized as capacitive, inductive, and conductive coupling. In this work, only the induced AC voltage on the pipelines due to inductive coupling in steady-state conditions is studied. A holistic mathematical model for the pipelines, power lines, mitigation equipment for the induced voltage, and cathodic protection (CP) is illustrated. Potassium hydroxide polarization cells are electrically represented because these cells are considered the most common mitigation device for discharging the induced AC voltage from the pipeline to the soil. The overall model is implemented using MATLAB. The results show the profiles of induced AC voltage along the pipeline, the CP for the pipeline, the points of maximum voltage, and the influence of installing the AC mitigation units on the CP performance.