Secure and Robust Demand Response Using Stackelberg Game Model and Energy Blockchain.

Demand response (DR) has been studied widely in the smart grid literature, however, there is still a significant gap in approaches that address security, privacy, and robustness of settlement processes simultaneously. The need for security and robustness emerges as a vital property, as Internet of Things (IoT) devices become part of the smart grid; in the form of smart meters, home energy management systems (HEMSs), intelligent transformers, and so on. In this paper, we use energy blockchain to secure energy transactions among customers and the utility. In addition, we formulate a mixed-strategy stochastic game model to address uncertainties in DR contributions of agents and achieve optimal demand response decisions. This model utilizes the processing hardware of customers for block mining, stores customer DR agreements as distributed ledgers, and offers a smart contract and consensus algorithm for energy transaction validation. We use a real dataset of residential demand profiles and photovoltaic (PV) generation to validate the performance of the proposed scheme. The results show the impact of electric vehicle (EV) discharging and customer demand reduction on increasing the probability of successful block mining and improving customer profits. Moreover, the results demonstrate the security and robustness of our consensus algorithm for detecting malicious activities.

Multi-Agent Team Learning in Virtualized Open Radio Access Networks (O-RAN).

Starting from the concept of the Cloud Radio Access Network (C-RAN), continuing with the virtual Radio Access Network (vRAN) and most recently with the Open RAN (O-RAN) initiative, Radio Access Network (RAN) architectures have significantly evolved in the past decade. In the last few years, the wireless industry has witnessed a strong trend towards disaggregated, virtualized and open RANs, with numerous tests and deployments worldwide. One unique aspect that motivates this paper is the availability of new opportunities that arise from using machine learning, more specifically multi-agent team learning (MATL), to optimize the RAN in a closed-loop where the complexity of disaggregation and virtualization makes well-known Self-Organized Networking (SON) solutions inadequate. In our view, Multi-Agent Systems (MASs) with MATL can play an essential role in the orchestration of O-RAN controllers, i.e., near-real-time and non-real-time RAN Intelligent Controllers (RIC). In this article, we first provide an overview of the landscape in RAN disaggregation, virtualization and O-RAN, then we present the state-of-the-art research in multi-agent systems and team learning as well as their application to O-RAN. We present a case study for team learning where agents are two distinct xApps: power allocation and radio resource allocation. We demonstrate how team learning can enhance network performance when team learning is used instead of individual learning agents. Finally, we identify challenges and open issues to provide a roadmap for researchers in the area of MATL based O-RAN optimization.

Energy-Aware Dynamic DU Selection and NF Relocation in O-RAN Using Actor-Critic Learning.

Open radio access network (O-RAN) is one of the promising candidates for fulfilling flexible and cost-effective goals by considering openness and intelligence in its architecture. In the O-RAN architecture, a central unit (O-CU) and a distributed unit (O-DU) are virtualized and executed on processing pools of general-purpose processors that can be placed at different locations. Therefore, it is challenging to choose a proper location for executing network functions (NFs) over these entities by considering propagation delay and computational capacity. In this paper, we propose a Soft Actor-Critic Energy-Aware Dynamic DU Selection algorithm (SA2C-EADDUS) by integrating two nested actor-critic agents in the O-RAN architecture. In addition, we formulate an optimization model that minimizes delay and energy consumption. Then, we solve that problem with an MILP solver and use that solution as a lower bound comparison for our SA2C-EADDUS algorithm. Moreover, we compare that algorithm with recent works, including RL- and DRL-based resource allocation algorithms and a heuristic method. We show that by collaborating A2C agents in different layers and by dynamic relocation of NFs, based on service requirements, our schemes improve the energy efficiency by 50% with respect to other schemes. Moreover, we reduce the mean delay by a significant amount with our novel SA2C-EADDUS approach.

Demand Management for Optimized Energy Usage and Consumer Comfort Using Sequential Optimization.

The Energy-efficiency of demand management technologies and customer's experience have emerged as important issues as consumers began to heavily adopt these technologies. In this context, where the electrical load imposed on the smart grid by residential users needs to be optimized, it can be better managed when customer's comfort parameters are used, such as thermal comfort and preferred appliance usage time interval. In this paper a multi-layer architecture is proposed that uses a multi-objective optimization model at the energy consumption level to take consumer comfort and experience into consideration. The paper shows how our proposed Clustered Sequential Management (CSM) approach could improve consumer comfort via appliance use scheduling. To quantify thermal comfort, we use thermodynamic solutions for a Heating Ventilation and Air Conditioner (HVAC) system and then apply our scheduling model to find the best time slot for such thermal loads, linking consumer experience to power consumption. In addition to thermal loads, we also include non-thermal loads in the cost minimization and the enhanced consumer experience. In this hierarchal algorithm, we classified appliances by their load profile including degrees of freedom for consumer appliance prioritization. Finally, we scheduled consumption within a Time of Use (ToU) pricing model. In this model, we used Mixed Integer Linear Programming (MILP) and Linear Programming (LP) optimization for different categories with different constraints for various loads. We eliminate the customer's inconvenience on thermal load considering ASHRAE standard, increase the satisfaction on EV optimal chagrining constrained by minimum cost and achieve the preferred usage time for the non-interruptible deferrable loads. The results show that our model is typically able to achieve cost minimization almost equal to 13% and Peak-to-Average Ratios (PAR) reduction with almost 45%.

A Survey on Recent Trends and Open Issues in Energy Efficiency of 5G.

The rapidly increasing interest from various verticals for the upcoming 5th generation (5G) networks expect the network to support higher data rates and have an improved quality of service. This demand has been met so far by employing sophisticated transmission techniques including massive Multiple Input Multiple Output (MIMO), millimeter wave (mmWave) bands as well as bringing the computational power closer to the users via advanced baseband processing units at the base stations. Future evolution of the networks has also been assumed to open many new business horizons for the operators and the need of not only a resource efficient but also an energy efficient ecosystem has greatly been felt. The deployment of small cells has been envisioned as a promising answer for handling the massive heterogeneous traffic, but the adverse economic and environmental impacts cannot be neglected. Given that 10% of the world's energy consumption is due to the Information and Communications Technology (ICT) industry, energy-efficiency has thus become one of the key performance indicators (KPI). Various avenues of optimization, game theory and machine learning have been investigated for enhancing power allocation for downlink and uplink channels, as well as other energy consumption/saving approaches. This paper surveys the recent works that address energy efficiency of the radio access as well as the core of wireless networks, and outlines related challenges and open issues.