Federated Deep Reinforcement Learning for User Access Control in Open Radio Access Networks

The Open Radio Access Network (O-RAN) introducing a particular unit known as RAN Intelligent Controllers (RICs) has been regarded as revolutionary paradigms to support multiclass wireless services required in the fifth and sixth generation (5G/6G) networks. Through unprecedentedly installing various machine learning (ML) algorithms to RICs, a RAN is able to intelligently configure resources/communications to support any vertical applications over any operating scenarios. However, to practically deploy this RAN paradigm, the O-RAN still suffers two critical issues of load balance and handover control, and therefore the very first ML algorithm for the O-RAN should effectively address these issues. In this paper, inspired by the superior performance of deep reinforcement learning (DRL) in tackling sequential decision-making tasks, we therefore develop an intelligent user access control scheme with the facilitation of deep Q-networks (DQNs). A federated DRL-based scheme is further proposed to train the parameters of multiple DQNs in the O-RAN, so as to maximize the long-term throughput and meanwhile avoid frequent user handovers with a limited amount of signaling overheads in the O-RAN. The simulation results have fully demonstrated the outstanding performance over the state-of-the-arts, to service the urgent needs in the standardization of the O-RAN.