| Flexible satellites in the form of multi-beams satellites can achieve higher-capacity broadband communication by flexibly distributing the payload on the satellite,thereby providing users with high-speed and highquality Internet services.As a key technology in flexible satellite communication systems,beam hopping technology has always been a research hotspot in the field of flexible satellite communication due to its characteristics of dynamically serving time-space uneven services and providing dynamic capacity.Faced with the complexity of the beam hopping problem and the continuous improvement of users’ requirements for the quality of communication services,how to intelligently implement dynamic beam scheduling is an urgent problem that needs to be solved at present.For dynamic beam scheduling in beam hopping,the traditional metaheuristic method can only provide the optimal decision at the current moment,ignoring the long-term benefits of the beam hopping problem,and needs to iterate in the huge beam hopping action space at each moment solution,resulting in a high time complexity.In this regard,dynamic beam scheduling scheme based on deep reinforcement learning can realize the optimization of long-term benefits of beam hopping problem,and can directly make decisions on the new state of the satellite according to the existing model,with extremely low time complexity.However,due to the limited on-board computing resources,it is difficult for online deep reinforcement learning to quickly learn the optimal strategy when dealing with complex satellite scenes,which are constantly changing dynamically,so usually only sub-optimal solutions can be obtained.Regarding the issues above,this paper deeply studies the beam hopping problem in the flexible satellite communication system,and constructs the beam hopping problem in large time-space scenarios as a sequential decision problem,taking into account the long-term benefit of system throughput and service fairness as the key performance indicators,and proposes a high-performance,lowcomplexity dynamic beam scheduling method,that is,a deep reinforcement learning powered meta-heuristic optimization method.This method uses the solution of the offline deep reinforcement learning model as the initial solution of the meta-heuristic method,and the meta-heuristic method continues to iteratively optimize to obtain the final beam hopping decision.Through simulation experiments verification,the proposed method not only improves the performance of deep reinforcement learning in the face of changing satellite scenes,but also reduces the large time complexity of the meta-heuristic method by reducing the search space of meta-heuristic method,while retaining the sequential decision-making advantages of deep reinforcement learning,and has better long-term performance than traditional meta-heuristics.Therefore,the proposed method can better optimize the long-term benefit of the system throughput and fairness with low time complexity,so as to meet the continuous improvement of users’ communication needs. |
