Load-Balanced Virtual Network Embedding Based on Deep Reinforcement Learning for 6G Regional Satellite Networks

Regional satellite networks are capable of supporting denser coverage and more reliable communications in the target area and hence have been viewed as an essential part of the sixth generation (6G) communication system. Since satellite networks are time-varying and have limited resources, efficient resource management schemes are needed to accommodate massive and ubiquitous service requests. As a remedy, virtual network embedding (VNE) can enable diverse virtual network requests (VNRs) to share the same substrate network resources to improve resource utilization. However, existing works are few and mainly rely on heuristic methods, whose static embedding strategies cannot be optimized according to the resource state. In this paper, we propose a deep reinforcement learning (DRL) aided load-balanced VNE algorithm (DRL-LBVNE) for the regional satellite networks, where we first build a low-cost regional satellite network scenario and derive its multi-fold coverage constraints. Besides, we design a novel preprocessing scheme to reduce mapping failure, where the satellite network is divided into multiple mapping regions, and VNRs are only deployed in the mapping region with the lowest load. In the node mapping stage, the DRL agent can calculate the embedding probabilities of each physical node based on the environment state. Moreover, a comprehensive metric for path selection is presented in the link mapping stage. Simulation results show that the DRL-LBVNE algorithm outperforms the other five state-of-art algorithms in acceptance rate, resource utilization, and average delay, reflecting better adaptability to dynamic satellite networks. © 2023 IEEE