Autonomous Exploration For Robots Via Deep Reinforcement Learning Based On Spatiotemporal Information On Graph

This thesis focuses on the problem of autonomous exploration in unknown environments for mobile robots,which is one of the key problems in robotics and is of great significance for improving the intelligence of robots and improving human production and life.Robots can assist and replace humans in hazardous and heavy tasks such as cave exploration,post-disaster rescue,and battlefield reconnaissance and further improve mission efficiency by using autonomous exploration technology.As a complete system,autonomous exploration usually consists of three parts: Simultaneous Localization and Mapping(SLAM),target point selection,path planning and navigation.This thesis mainly focuses on the target point selection strategy based on Spatiotemporal Neural Network on Graph(Graph-STNN)and deep reinforcement learning(DRL).The main work and innovation of this thesis are as follows:This thesis investigates the problem of single-robot autonomous exploration based on DRL under spatiotemporal information on the graph,improving the efficiency of robot exploration in unknown environments and increasing the exploration algorithm’s scalability for different environments.The exploration environment is described as a graph in non-Euclidean space,and a novel Graph-STNN is designed to extract temporal and spatial information from the exploration graph.In order to eliminate the need to design complex target point evaluation algorithms artificially,we construct a DRL-based autonomous exploration framework based on the extracted spatiotemporal information.It takes unsupervised training to learn the evaluation model of target points.The comparison experiments with information entropy-based and GCN-based algorithms in two-dimensional grid environments demonstrate that the proposed method in this thesis has higher exploration efficiency and the role of temporal and spatial information in improving the speed and accuracy of exploration.Furthermore,this thesis also conducts model pre-training in the robot simulation platform(Stage)and migrates the whole system to the physical platform for testing,verifying the feasibility of the proposed algorithm.Based on single-robot autonomous exploration,we investigate the problem of multirobot autonomous exploration based on DRL under spatiotemporal information on the graph,and the algorithm is scalable to the number of robots.This thesis uses a multirobot distributed exploration architecture to design a DRL framework for collaborative multi-robot exploration.Then,a distributed exploration graph is designed to describe the environment.Next,a distributed exploration network is designed to fuse the information of each robot’s trajectory and the spatial structure of the whole environment.This thesis describes multi-robot distributed autonomous exploration as a traveling salesman problem and constructs a cost matrix based on it.Then the optimal transmission theory is used to solve the minimum cost of the whole system,which is incorporated into the reward function to guide the exploration of the whole multi-robot system.Finally,the model is trained and tested in the Stage to verify the effectiveness of the multi-robot distributed exploration algorithm and the robustness to the number of robots.