RIS-assisted UAV Networks: Deployment Optimization with Reinforcement-Learning-Based Federated Learning

Unmanned Aerial Vehicles assisted (UAV-assisted) communication with Reconfigurable Intelligent Surfaces (RIS) is one of the key technologies for future 6G communication due to the advantages, such as high mobility, coverage extend, power-saving, and signal concentration. However, the deployment of UAVs to optimize the overall objective of the system is proved to be an NP-hard problem. To address the complexity issue, several approaches present heuristic algorithms as a solution. Nevertheless, the request to locate the users for heuristic algorithms can lead to an invasion of privacy. In this paper, we propose a reinforcement learning solution with the federated learning framework, each UAV as an agent with Q-table to learn the deployment via observation from the total capacity of the relayed users. In addition, UAVs update the Q-table values through federated learning with others to bring the learning process together. As UAVs do not require location information and only model parameters are exchanged, user privacy is protected. The simulation shows that the proposed method deploys the UAVs with less information obtained in advance, and reaches the ability to transmit to the users as high as possible.