The Research On Path Optimization For UAV Rescue

In recent years,UAV has been widely used in post disaster rescue.It is very important to plan efficient and reasonable rescue path for UAV in uncertain environment,which not only helps to reduce flight time and increase UAV utilization,but also improves rescue efficiency.In this paper,the real problem of UAV rescue in uncertain environment is studied,and the dynamic process of UAV rescue is analyzed.Based on the priority division of fuzzy clustering,the rescue path planning model of UAV Partially Observable Markov Decision Processes(POMDP)is builded,and the point based approximation algorithm ? Successive Approximations of the Reachable Space under Optimal Policies is introduced to solve the POMDP model.And simulation experiments are carried out to verify the proposed optimization model and algorithm.The main contents of this paper are as follows:First of all,this paper makes an in-depth study on the current situation of UAV and the path optimization of rescue UAV,and finds that the simulation of real scene is not perfect in the current research on rescue problem modeling.Considering the uncertainty of the environment of the system and the uncertainty of the action transfer in the real rescue,this paper introduces the Markov Decision Process(MDP)and the Partially Observable Markov Decision Process(POMDP)theory in the reinforcement learning field to solve the uncertainty problem,makes a systematic summary of the MDP and POMDP theory,and provides theoretical basis for the study.Secondly,on the basis of POMDP theory,the rescue space is mapped to twodimensional plane,and based on POMDP model elements,the UAV rescue path planning model is builded.Based on the index of regional rescue demand,this paper uses the method of fuzzy clustering to classify the rescue areas,and calculates the average utility value to measure the priority degree of each cluster.According to the actual problems,the rescue target area is divided into cell grids and state elements are builded.The conventional 8 directions are selected as the action elements,whether the victims are observed as the observation elements,the transfer probability function and observation function are set according to the actual situation,the reward function is set according to the priority degree of the state.Then,aiming at the UAV rescue model based on priority division,this paper introduces a point based approximation algorithm-Successive Approximations of the Reachable Space under Optimal Policies(SARSOP),which is divided into three steps: search,update and prune.A set of representative points in belief space is obtained by sampling with selective depth search as the approximate representation of belief space,that is,the Optimally Reachable Belief Space.Then,the belief state and vector set are updated by backup operation,and the invalid vectors are pruned to improve the efficiency of calculation strategy.Finally,based on the POMDP model of UAV rescue and SARSOP algorithm,simulation experiments are carried out.Simulation experiment 1 mainly shows the process of UAV's self-learning and cognition,the dynamic change process from uncertain environment state to final goal.Simulation experiment 2 mainly shows the state changes of POMDP strategy,greedy strategy and fixed strategy from the perspective of God,and verifies that POMDP strategy is the fastest strategy to reach the high priority state.Simulation experiment 3 divides the experiment into three kinds of scenes(optimistic placement scenario,mixed placement scenario and pessimistic placement scenario).Taking the time to find the victims as the measurement index,comparing POMDP strategy and greedy strategy,POMDP strategy can always find the victims in different placement scenarios quickly.Simulation experiment 4,as an extension experiment,aims to make the UAV avoid obstacles and reach the target point accurately(precise delivery).The POMDP strategy is compared with the strategy based on potential energy field to verify that the POMDP strategy is less limited.The experimental results verify that the POMDP model proposed in this paper can dynamically intervene every step of path planning in uncertain environment,and verify the effectiveness of SARSOP algorithm in solving practical problems,which has a certain practical value in the field of UAV rescue path optimization.