Research On Charging Strategy Of Air Terrestrial IoT Network Based On Deep Reinforcement Learning

In recent years,with the vigorous development of the industrial Internet,the Internet of Things technology has become an important support to drive the development of the digital society.The Internet of Things technology is widely used in industry,agriculture,service industry and other industries.It plays an important role in promoting digital transformation and achieving high-quality development.The application of the Internet of Things system to the field environmental monitoring has become the current research focus,because the environmental monitoring through wireless terminals such as sensors has the advantages of high accuracy,strong flexibility,low cost and other traditional monitoring methods can not match.Although the Internet of Things system has been relatively mature in the indoor environment or urban environment with stable power supply,there are still problems of low sustainability and poor stability in the outdoor environment.On the one hand,due to its small size,the battery capacity carried by the sensor is limited,which cannot support long-term work.In addition,the environment in the field is more complex,and the cost of manual battery replacement is high.On the other hand,due to the poor infrastructure in the field area,it is impossible to charge the sensor nodes in an active way.In view of the above problems,this paper adopts the combination of wireless charging sensor and long-distance wireless charging,that is,integrate the UAV with wireless charging equipment into the terrestrial network to transmit power for the sensor.This thesis mainly studies the charging strategy in the air-terrestrial IoT network.The specific research contents are as follows:(1)Research on path planning of UAV charging mission in air-terrestrial IoT network.The goal of this problem is to maintain the sustainability of the entire terrestrial network and minimize the total energy consumption of the UAV by reasonably planning the charging path of the UAV.This thesis uses deep reinforcement learning to solve the problem,designs appropriate state,action space and reward function according to the particularity of UAV wireless charging task,and converts path optimization task into sequence generation task.The deep reinforcement learning algorithm of coder-decoder structure is used to complete the path planning,and the encoder part adopts the attention mechanism;The decoder part uses multiple decoders to perform parallel decoding at the same time,and finally selects the path with the best effect as the output result.(2)Research on the sustainability of UAV mission execution in air terrestrial IoT network.In the first research content,the sustainability of the ground sensor network is realized through the charging path planning of the UAV,but the UAV needs to return frequently to the takeoff node for energy supplement during the task execution process,which does not realize the sustainability of the UAV task execution process.Therefore,this thesis then carries out the research on the sustainability of UAV mission execution in the air-ground IoT network.By designing a reasonable UAV periodic charging strategy,it not only realizes the sustainability of the ground network,but also realizes the sustainability of UAV mission execution.In this thesis,a two-stage UAV periodic charging strategy based on DQN is proposed.Firstly,the sensor node to be visited is selected according to the probability result calculated by neural network,and then the charging amount of the corresponding node is obtained according to the designed charging strategy function.The UAV receives rewards based on the interaction between the access node and the charging volume and the environment,and finally obtains the optimized charging scheduling strategy through continuous training.