Path Planning Method Based On Deep Reinforcement Learning And Its Application

In recent years,crowd evacuation simulation has become a research hotspot in the field of public security.Especially in the public places where the crowd is concentrated,the indecision and panic of the individual in the emergency will lead to potential safety hazards such as crowd pushing and trampling on others.Therefore,it is very important to plan the escape route reasonably.In order to ensure the rationality of the escape route,it is necessary for the crowd to conduct the evacuation drill.Traditional crowd evacuation exercise is time-consuming and hard to plan the best evacuation path.More and more scholars begin to pay attention to crowd evacuation simulation and modeling.The computer simulation technology uses the form of deduction to explore the path rule in crowd evacuation,provides theoretical guidance for architectural design and emergency management,and plays a significant role in preventing public accidents and ensuring public safety.Therefore,it is necessary to use computer simulation method to study the evacuation of public places in an emergency.It is a key problem to make a reasonable path planning for pedestrian movement to avoid destructive behavior in crowd evacuation simulation.At present,most of the path planning methods in practical application are often faced with complex environment,low efficiency and complex calculation,especially when applied to crowd evacuation simulation,the phenomenon of aggregation and grouping between pedestrians and the movement details in the evacuation process are ignored,such as the formation of pedestrian queue,exit selection,etc.Therefore,this paper proposes an efficient path planning method based on deep reinforcement learning and applies it to crowd evacuation simulation.Firstly,a multi-agent reinforcement learning model based on congestion detection is proposed,which fully considers the "aggregation-grouping" effect in real life,and defines reinforcement learning elements on the basis of detecting the degree of outlet congestion,which can well reflect the impact of crowd relationship and outlet congestion on path planning,and make the evacuation efficiency more efficient.In addition,a path planning algorithm based on deep reinforcement learning is proposed.The upper layer usesthe improved multi-agent deep deterministic policy gradient(IMADDPG)algorithm to plan the macro path,and the lower layer's reciprocal velocity obstacle(RVO)to achieve collision avoidance and group following.Finally,this method is applied to the simulation of crowd evacuation,and different experiments show that the above methods can improve the efficiency of path finding and evacuation,and provide visual analysis and theoretical guidance for building design,disaster emergency management,etc.The main work and innovation of this paper are as follows:(1)In view of the existing evacuation studies,most of them ignore crowd aggregation,grouping and exit selection,a multi-agent reinforcement learning model based on congestion detection is proposed.In the first step,a group computing method based on K-means algorithm is proposed,which fully considers the "aggregation-grouping" effect in real life,groups people according to the distance between individuals,selects the group leader according to the cluster center,and reappears the "aggregation-grouping" phenomenon in the crowd;in the second step,the leader is modeled as an agent;in the third step,by comparing the speed value of the leader agent in the congestion detection area,the congestion situation at the exit is determined,and the congestion situation is designed into the reward function of reinforcement learning.The model can well reflect the impact of crowd relationship and the degree of exit congestion on the path planning,realize the exit selection of pedestrians in the scene,and make the evacuation efficiency more efficient.(2)In order to solve the problems of low efficiency and complex calculation when the traditional path planning algorithm is applied to complex scenes,a path planning method based on IMADDPG algorithm is proposed.Based on the existing multi-agent deep deterministic policy gradient(MADDPG)algorithm,IMADDPG adds additional new networks.In the new network,the mean field theory is used to reduce the complexity of additional training samples.It can better complete the task of collaborative global path planning.In this method,IMADDPG algorithm is executed in the upper layer to train the global path for the leader agent,and the trained path is shared with the followers in the group;RVO algorithm is executed in the lower layer to realize path following and local collision avoidance for the followers.This methodimproves the group's perception of environmental information,and more efficiently plans the evacuation path for pedestrians in crowd evacuation simulation.(3)Build a three-dimensional evacuation simulation platform.The platform integrates simulation control function,camera control function and rendering output function.By analyzing the results of realistic rendering,we can see that the path planning method proposed in this paper is more efficient when applied to crowd evacuation simulation,to achieve rapid pedestrian evacuation,to simulate exit selection and crowd queue movement behavior.The simulation effect is of great significance for guiding and studying crowd evacuation.