Multi-Tier Deep Reinforcement Learning for Non-Terrestrial Networks

Yang Cao, Shao Yu Lien, Ying Chang Liang, Dusit Niyato

Research output: Contribution to journalArticlepeer-review


To provide global coverage and ubiquitous wireless services, non-terrestrial networks (NTNs) composed of space-tier, air-tier, and ground-tier stations, have been regarded as a key enabling technology toward the sixth generation (6G) networks. Such a multi-tier architecture, however, induces fundamental challenges in optimizing the overall system performance of the NTN in terms of signaling overheads, complexity, and even analytical formulation of the optimization problem. Interestingly, deep reinforcement learning (DRL) is an effective approach to tackle "complex" optimization problems. When applied to the NTN, however, different DRL methods should be developed for various stations due to different constraints in each tier, which raises fundamental issues involving the interaction, orchestration, and protocol design of multi-tier DRL architecture. In this article, we provide the essential principles for multi-tier DRL for the NTN, and propose a generalized multi-tier architecture to embrace different deployment scenarios of NTNs. A case study of joint optimization of spectrum chunk allocation, trajectory design, and user association for space-tier, air-tier, and ground-tier stations is provided. The simulation results indicate that different trade-offs between the overall throughput, optimization dimension, and computational complexity of stations can be achieved through configuring the proposed architectures.

Original languageEnglish
Pages (from-to)1-8
Number of pages8
JournalIEEE Wireless Communications
StateAccepted/In press - 2024


  • Computer architecture
  • Optimization
  • Orbits
  • Space vehicles
  • Task analysis
  • Training data
  • Wireless communication


Dive into the research topics of 'Multi-Tier Deep Reinforcement Learning for Non-Terrestrial Networks'. Together they form a unique fingerprint.

Cite this