| Currently,many large technology companies around the world are actively researching and developing low-earth-orbit satellite network technology,including Space X,One Web,Amazon,etc.This is due to the characteristics of low-earth-orbit satellites,such as low orbital altitude and support for multiple application scenarios.However,when facing high traffic in densely populated areas,there may be problems with link congestion leading to a decrease in data transmission efficiency.Reinforcement learning technology can continuously update strategies in real-time interactions,and has strong adaptability and learning ability.Its technical advantage lies in the ability to update strategies in real-time and adapt to complex and changing environments.This thesis applies reinforcement learning to solve the problem of load balancing in low-earth-orbit satellite network routing,using its adaptability to focus on the problem of unbalanced traffic load in low-earth-orbit satellite networks,and proposes two new routing schemes:(1)Load Balancing Routing Algorithm Based on Q-routing in Low-Earth-Orbit Satellite Network.This algorithm introduces randomness in action selection based on the Q-routing algorithm and sets the remaining cache as a load evaluation when updating Q-values.It also adds policy gradient calculation to dynamically adjust the learning rate,which improves the performance of the routing algorithm to some extent,reduces the risk of network congestion,and balances the network load.(2)Ant Colony Optimization Algorithm Based on Reinforcement Learning in Low-EarthOrbit Satellite Network.This algorithm introduces load-aware heuristic information based on the traditional ant colony routing mechanism and designs a reward function for link information evaluation.By considering the current node’s transmission cost,time delay,and load status,and combining these with pheromones,it improves the updating rules of pheromones and fully exerts the directional effect of pheromones,selects lightly loaded candidate nodes as the next hop more accurately,and effectively improves the algorithm’s load balancing ability and reliability.This thesis evaluates the routing performance of the proposed algorithm using the NS2 simulation platform.The simulation results verify the correctness and effectiveness of the proposed algorithm and demonstrates its adaptability to the characteristics of low-earth-orbit satellite networks and the ability to balance network loads,effectively improving network performance. |
