On the Design of a Network Digital Twin for the Radio Access Network in 5G and Beyond.

A Network Digital Twin (NDT) is a high-fidelity digital mirror of a real network. Given the increasing complexity of 5G and beyond networks, the use of an NDT becomes useful as a platform for testing configurations and algorithms prior to their application in the real network, as well as for predicting the performance of such algorithms under different conditions. While an NDT can be defined for the different subsystems of the network, this paper proposes an NDT architecture focusing on the Radio Access Network (RAN), describing the components to represent and model the operation of the different RAN elements, and to perform emulations. Different application use cases are identified, and among them, the paper puts the focus on the training of Reinforcement Learning (RL) solutions for the RAN. For this use case, the paper introduces a framework aligned with O-RAN specifications and discusses the functionalities needed to integrate the NDT. This use case is illustrated with the description of a RAN NDT implementation used for training an RL-based capacity-sharing solution for network slicing. Presented results demonstrate that the implemented RAN NDT is a suitable platform to successfully train the RL solution, achieving service-level agreement satisfaction values above 85%.

A Deep Q-Network-Based Algorithm for Multi-Connectivity Optimization in Heterogeneous Cellular-Networks.

The use of multi-connectivity has become a useful tool to manage the traffic in heterogeneous cellular network deployments, since it allows a device to be simultaneously connected to multiple cells. The proper exploitation of this technique requires to adequately configure the traffic sent through each cell depending on the experienced conditions. This motivates this work, which tackles the problem of how to optimally split the traffic among the cells when the multi-connectivity feature is used. To this end, the paper proposes the use of a deep reinforcement learning solution based on a Deep Q-Network (DQN) in order to determine the amount of traffic of a device that needs to be delivered through each cell, making the decision as a function of the current traffic and radio conditions. The obtained results show a near-optimal performance of the DQN-based solution with an average difference of only 3.9% in terms of reward with respect to the optimum strategy. Moreover, the solution clearly outperforms a reference scheme based on Signal to Interference Noise Ratio (SINR) with differences of up to 50% in terms of reward and up to 166% in terms of throughput for certain situations. Overall, the presented results show the promising performance of the DQN-based approach that establishes a basis for further research in the topic of multi-connectivity and for the application of this type of techniques in other problems of the radio access network.