Research On Target Driven Robot Environment Exploration Algorithm Based On Deep Reinforcement Learning

Robot autonomous environment exploration system aims to independently exploring unknown or partially known environments,extracting information about the surroundings,and making decisions based on that information.When faced with exploration tasks in unknown environments,goal-driven robots need to make trade-offs between reaching the global goal and conducting exploration,in order to efficiently accomplish the task.This thesis presents a new reinforcement learning algorithm designed to overcome the computational inefficiency and susceptibility to local optimal issues of traditional reinforcement learning methods in robot environment exploration.It addresses the task of autonomous exploration in unknown robot environments.The main contributions of this thesis are summarized as follows:(1)This research develops a deep reinforcement learning-based robot target-driven autonomous environment exploration system after comparing existing autonomous environment exploration techniques for robotics.The global navigation system and the local navigation system constitute the majority of the system.Firstly,this thesis designs a modular architecture for autonomous robot environment exploration system.The robot autonomous environment exploration system uses laser radar as sensors to collect environmental information and construct an occupation grid map.Using heuristic functions and improved deep reinforcement learning algorithms to achieve robot navigation and explore maps.(2)In global navigation,this thesis uses heuristic functions based on graph search and relaxation techniques to design a new heuristic function that uses the relative relationship between robot coordinates and target point coordinates,as well as environmental information,to evaluate potential path points existing in exploration tasks,and select intermediate path points.The navigation task of the robot is performed by a deep neural network.The calculation of the heuristic function considers the training settings of the network based on deep reinforcement learning,and combines it with distance information and map information pointing to the global target.Further,the results calculated by the heuristic function include the coordinates of the intermediate path points,which will also be used as input information for the deep network.(3)In the research of deep reinforcement learning-based local navigation,the issue that conventional deep reinforcement learning algorithms can only be used to a single navigation goal was addressed.This thesis models the robot local navigation problem as a goal driven Markov decision-making process,solving the problem that classical deep reinforcement learning algorithms need to relearn strategies for different navigation targets.In addition,considering the smoothness of the path and collision avoidance,this thesis designs a new reward function to optimize the low efficiency of deep reinforcement learning training and the possible local optimization problems.(4)To address the issues of partial environmental observations caused by environmental occlusion and sensor limitations in the context of robot local navigation,this thesis proposes a deep reinforcement learning approach with enhanced historical memoryand introduces the combination of the Deep Deterministic Policy Gradient(DDPG)algorithm and the Proximal Policy Optimization(PPO)algorithm with Long Short-Term Memory(LSTM)networks..Based on an improved Long Short-Term Memory network,this thesis designs a memory structure that includes spatial memory and action memory.By combining it with classic deep reinforcement learning algorithms,temporal dependencies in navigation problems can be learned.This improves the performance and robustness of autonomous robot exploration in time series decisionmaking scenarios for local navigation,and enhances the ability of robot state estimation in partially measurable environments.