Towards Stochastic Optimization-Based Electric Vehicle Penetration in a Novel Archipelago Microgrid.

Due to the advantage of avoiding upstream disturbance and voltage fluctuation from a power transmission system, Islanded Micro-Grids (IMG) have attracted much attention. In this paper, we first propose a novel self-sufficient Cyber-Physical System (CPS) supported by Internet of Things (IoT) techniques, namely "archipelago micro-grid (MG)", which integrates the power grid and sensor networks to make the grid operation effective and is comprised of multiple MGs while disconnected with the utility grid. The Electric Vehicles (EVs) are used to replace a portion of Conventional Vehicles (CVs) to reduce CO 2 emission and operation cost. Nonetheless, the intermittent nature and uncertainty of Renewable Energy Sources (RESs) remain a challenging issue in managing energy resources in the system. To address these issues, we formalize the optimal EV penetration problem as a two-stage Stochastic Optimal Penetration (SOP) model, which aims to minimize the emission and operation cost in the system. Uncertainties coming from RESs (e.g., wind, solar, and load demand) are considered in the stochastic model and random parameters to represent those uncertainties are captured by the Monte Carlo-based method. To enable the reasonable deployment of EVs in each MGs, we develop two scheduling schemes, namely Unlimited Coordinated Scheme (UCS) and Limited Coordinated Scheme (LCS), respectively. An extensive simulation study based on a modified 9 bus system with three MGs has been carried out to show the effectiveness of our proposed schemes. The evaluation data indicates that our proposed strategy can reduce both the environmental pollution created by CO 2 emissions and operation costs in UCS and LCS.

Multistep Multiagent Reinforcement Learning for Optimal Energy Schedule Strategy of Charging Stations in Smart Grid.

An efficient energy scheduling strategy of a charging station is crucial for stabilizing the electricity market and accommodating the charging demand of electric vehicles (EVs). Most of the existing studies on energy scheduling strategies fail to coordinate the process of energy purchasing and distribution and, thus, cannot balance the energy supply and demand. Besides, the existence of multiple charging stations in a complex scenario makes it difficult to develop a unified schedule strategy for different charging stations. In order to solve these problems, we propose a multiagent reinforcement learning (MARL) method to learn the optimal energy purchasing strategy and an online heuristic dispatching scheme to develop a energy distribution strategy in this article. Unlike the traditional scheduling methods, the two proposed strategies are coordinated with each other in both temporal and spatial dimensions to develop the unified energy scheduling strategy for charging stations. Specifically, the proposed MARL method combines the multiagent deep deterministic policy gradient (MADDPG) principles for learning purchasing strategy and a long short-term memory (LSTM) neural network for predicting the charging demand of EVs. Moreover, a multistep reward function is developed to accelerate the learning process. The proposed method is verified by comprehensive simulation experiments based on real data of the electricity market in Chicago. The experiment results show that the proposed method can achieve better performance than other state-of-the-art energy scheduling methods in the charging market in terms of the economic profits and users' satisfaction ratio.

A leader-following paradigm based deep reinforcement learning method for multi-agent cooperation games.

Multi-agent deep reinforcement learning algorithms with centralized training with decentralized execution (CTDE) paradigm has attracted growing attention in both industry and research community. However, the existing CTDE methods follow the action selection paradigm that all agents choose actions at the same time, which ignores the heterogeneous roles of different agents. Motivated by the human wisdom in cooperative behaviors, we present a novel leader-following paradigm based deep multi-agent cooperation method (LFMCO) for multi-agent cooperative games. Specifically, we define a leader as someone who broadcasts a message representing the selected action to all subordinates. After that, the followers choose their individual action based on the received message from the leader. To measure the influence of leader's action on followers, we introduced a concept of information gain, i.e., the change of followers' value function entropy, which is positively correlated with the influence of leader's action. We evaluate the LFMCO on several cooperation scenarios of StarCraft2. Simulation results confirm the significant performance improvements of LFMCO compared with four state-of-the-art benchmarks on the challenging cooperative environment.