Multiagent Q-Learning-Based Mobility Management for Multi-Connectivity in mmWAVE Cellular Systems.

Effective mobility management is crucial for efficient operation of next-generation cellular systems in the millimeter wave (mmWave) band. Massive multiple-input-multiple-output (MIMO) systems are seen as necessary to overcome the significant path losses in this band, but the highly directional beam makes the channels more susceptible to radio link failures due to blockages. To meet stringent capacity and reliability requirements, multi-connectivity has attracted significant attention. This paper proposes a multiagent distributed Q learning-based mobility management scheme for multi-connectivity in mmWave cellular systems. A hierarchical structure is adopted to address the model complexity and speed up the learning process. The performance is assessed using a realistic measurement data set collected from Wireless Insite in an urban area and compared with independent Q learning and a heuristic scheme in terms of handover probability and spectral efficiency.

Distributed Spectrum Management in Cognitive Radio Networks by Consensus-Based Reinforcement Learning.

In this paper, we propose a new algorithm for distributed spectrum sensing and channel selection in cognitive radio networks based on consensus. The algorithm operates within a multi-agent reinforcement learning scheme. The proposed consensus strategy, implemented over a directed, typically sparse, time-varying low-bandwidth communication network, enforces collaboration between the agents in a completely decentralized and distributed way. The motivation for the proposed approach comes directly from typical cognitive radio networks' practical scenarios, where such a decentralized setting and distributed operation is of essential importance. Specifically, the proposed setting provides all the agents, in unknown environmental and application conditions, with viable network-wide information. Hence, a set of participating agents becomes capable of successful calculation of the optimal joint spectrum sensing and channel selection strategy even if the individual agents are not. The proposed algorithm is, by its nature, scalable and robust to node and link failures. The paper presents a detailed discussion and analysis of the algorithm's characteristics, including the effects of denoising, the possibility of organizing coordinated actions, and the convergence rate improvement induced by the consensus scheme. The results of extensive simulations demonstrate the high effectiveness of the proposed algorithm, and that its behavior is close to the centralized scheme even in the case of sparse neighbor-based inter-node communication.