Deep Reinforcement Learning-Based Routing for Space-Terrestrial Networks

Satellite communication is one primary structure of the future wireless systems. Many corporations and academic institutions are committed to this research, e.g., StarLink. Their aim is to construct a constellation of satellite networks covering the whole world. Thanks to this design, our mobile devices can connect to the Internet anywhere in the world without being restricted by the coverage area of base stations. However, given that the constellation of satellites around the Earth is a system that varies in time, algorithms must adapt to the dynamic topology. To solve the problem, in this paper we investigate how to send data from the base station to the target satellite with minimal delay. We systematically formulate the transmission limitations, uplink, and downlink transmission delay. With well defined limits and delays on space-terrestrial transmissions, we can set rewards and punishments by mathematical analysis to simulate the agent in deep reinforcement learning (DRL), which can explore all choices in the system and optimize the routing algorithm. Finally, the simulation results corroborate a fully functional space-terrestrial network constellation to simulate the actual situation.