Research On Crowd Movement Behavior Modeling Method Based On Deep Reinforcement Learning

Due to the increasing population,it is easy to gather a large number of people in some public places.In case of fire,collapse and other emergencies in places with dense crowd flow,it is necessary to evacuate the personnel in the place in time.Previously,some scholars organized a certain number of people to conduct evacuation drills in a specific place,but this method requires a lot of manpower and material resources and is not easy to reproduce.With the development of computer simulation technology,some scholars began to use computer simulation technology to simulate the process of crowd evacuation in the field of crowd evacuation.Compared with the traditional methods of organizing personnel to conduct evacuation exercises,crowd evacuation simulation method can save human and financial resources,and has the advantages of high reproducibility and real-time.Therefore,using the crowd evacuation simulation method to simulate the movement of the crowd in the designated place can observe the crowd movement law,provide a reference for the formulation of the evacuation plan,and is of great significance to protect the safety of people’s lives and property in an emergency.The traditional crowd evacuation simulation methods are mostly based on artificial constraints and assumptions,so the simulated crowd movement is inconsistent with the reality,and the simulation reality is low.The data-driven crowd evacuation method simulates the movement of the crowd according to the pedestrian trajectory extracted from a specific scene,but the crowd movement cannot be adjusted when the scene changes,and the model lacks flexibility and generalization.A key aspect of crowd evacuation process is individual decision-making.Determining the optimal evacuation strategy at the individual level,that is,an optimal path,can improve the evacuation efficiency.Due to the lack of consideration and design of exit congestion,exit selection and other movement behaviors in the current path planning methods used in the field of crowd evacuation,the problems such as low evacuation efficiency and easy to fall into local optimization are caused.Based on the deep reinforcement learning algorithm,this thesis focuses on how to overcome the problem that the data-driven method can’t adapt to the dynamic changes of the scene and how to improve the efficiency of crowd evacuation.According to the problems in the above crowd evacuation methods,this thesis proposes a crowd movement behavior modeling method based on deep reinforcement learning.The innovative research results obtained can be summarized as follows:(1)Aiming at the problem that the data-driven crowd evacuation method can’t adapt to the dynamic changes of the scene,an adaptive crowd motion modeling method based on double-layer deep reinforcement learning is proposed in this thesis.Firstly,a hierarchical deep reinforcement learning framework divided into macro control layer and micro control layer is established to deal with path planning and collision avoidance respectively.Secondly,a path planning method integrating data-driven and deep reinforcement learning is proposed.This method adds the distance between the agent and its surrounding obstacles to the state space,so that the agent can perceive the changes of the surrounding environment in real time,improve the adaptability of the agent to the scene,and simulate the pedestrian trajectory in the video.Finally,a local collision avoidance algorithm based on Multi-Agent Deep Deterministic Policy Gradient(MADDPG)is implemented,and a return function combined with Relative Velocity Obstacles(RVO)algorithm is designed to improve the computational efficiency of generating collision free velocity.The experimental results show that the modeling method can improve the flexibility of the model.(2)In view of the lack of consideration and design of exit congestion,exit selection and other movement behaviors in the current path planning methods used in the field of crowd evacuation,a crowd congestion behavior modeling method based on deep reinforcement learning is proposed in this thesis.Firstly,a congestion avoidance path planning method based on deep reinforcement learning is proposed.The congestion degree at the exit is judged by analyzing the relationship between crowd density and traffic coefficient,so as to maximize the utilization of the exit.Secondly,a collision avoidance algorithm integrating video data and deep reinforcement learning is proposed.Pedestrian collision avoidance data,such as relative position and speed,are extracted from the video data,and the corresponding reward function is designed to make the crowd’s obstacle avoidance behavior more real in the case of congestion.Finally,compared with other evacuation simulation methods,the effectiveness of the proposed method is verified.The experimental results show that this method can improve the crowd evacuation efficiency and shorten the crowd evacuation time.(3)This thesis constructs a three-dimensional crowd evacuation navigation system integrating online modeling,real-time navigation and rendering output functions.The system is based on Windows 10 operating system and is based on unity 2020 3.8 for the development environment,the methods proposed in Chapter 3 and Chapter 4 are used in the simulation process.Simulation results show that the proposed method can improve the evacuation authenticity,enhance the generalization of the model and improve the evacuation efficiency.Finally,it introduces that the system has practical functions such as navigation and online modeling,and uses the system on Android platform.