Research On Mobile Robot Navigation Based On Deep Reinforcement Learning In Human-Robot Integration Environment

As robots are increasingly used in human-centric environments,reaching target locations safely and efficiently in crowded environments is critical for mobile service robots.In crowded spaces,dynamic pedestrians interact with each other implicitly.The robot cannot observe the pedestrian’s intended target and preferred walking style,which makes crowd navigation difficult.How to make robots comply with the social norms of human walking and respect the behavioral intentions of pedestrians while efficiently planning collision-free paths is an important problem in the field of mobile robotics research,with significant practical significance and research value.To address the above issues,this paper explores and investigates robot fusion localization,human-robot interaction modeling and deep reinforcement learning navigation methods,which are mainly as follows:First,the Error-State Kalman Filter(ESKF)localization method based on the fusion of Inertial Measurement Unit(IMU)and Li DAR information is proposed for the problem of mobile robots requiring high accuracy localization in a crowded environment.The IMU data is integrated into a nominal state that accumulates errors and needs to fuse information from the Li DAR to correct the results of the IMU motion integral.The error containing all noise and perturbations is collected in the error state and estimated using the ESKF,which performs filtering corrections as the lidar information arrives.The estimated states are recursively corrected by generating new features in each ESKF iteration,keeping the system computationally tractable.Simulation experimental results show that the proposed method meets the requirements for high-precision localization of mobile robots.Second,a Convolutional Neural Network(CNN)-based spatio-temporal graph framework for capturing spatio-temporal human-machine interaction information is proposed for modeling human-machine interaction in crowd navigation.In the spatial dimension,a graph attention network based on spatial features with jump connections is used and the spatial interaction between human and robot is modeled through a gating mechanism.The graph neural network is used to capture the spatial interaction features of robot and pedestrian as well as the spatial relative relationships.In the temporal dimension,a modified time-domain convolutional network is used to capture the temporal dynamics of the robot and the pedestrian.The gated highway mechanism of CNN is introduced to dynamically regulate the information flow of human-robot interaction by focusing on more salient features,while the feed-forward nature of CNN allows the model to achieve higher efficiency and more accurate predictions in training and inference.Experimental results show that the proposed model has more accurate and efficient modeling of interactions between robots and pedestrians within and across each time step.Finally,a learning framework for crowd navigation of mobile robots based on model-free deep reinforcement learning and human-robot spatio-temporal interaction models is proposed for the navigation problem of mobile robots that need to reach the target location smoothly and efficiently in a crowd environment.In this paper,we consider encoding the crowd and robot interactions into the navigation policy,decomposing the complex problem into smaller factors by identifying independent components of robot crowd navigation,converting the spatio-temporal graph for crowd navigation scene modeling into a new end-to-end decentralized structured network,and finally combining with Recurrent Neural Network(RNN)to efficiently learn the parameters of the corresponding factors.The use of model-free reinforcement learning to directly learn the navigation policy avoids premature convergence of the network to a suboptimal policy.Simulation experimental results show that the proposed method satisfies the requirements for the robot to reach the target location smoothly and efficiently in a crowd.