Research On Multi-modal Perception And Deep Reinforcement Learning Based Local Path Planning

In recent years,artificial intelligence technology has achieved great success in many fields,and local path planning methods based on deep learning(DL)or deep reinforcement learning(DRL)have been extensively developed.Deep learning-based methods usually require a large number of manual annotations or expert demonstrations,and the learned actions are usually limited to the data set,and the generalization ability is poor.The existing DRL-based methods are mainly developed in simulation environments,and there is very little work which can be finally deployed using a real robot.In this thesis,we present a novel local path planning method for unmanned ground vehicle(UGV)based on deep reinforcement learning(DRL).In this thesis,we propose a novel DRL-based local path planner.The planner decouples environment perception and decision-making,and achieves reliable online interaction with the environment through image-lidar multi-modal perception.Based on the interaction,the planner can directly implement the policy learning which generates flexible actions to avoid collisions with obstacles during driving.In perception,to narrow the gap between simulation and the real environment,we use the intermediate representation-semantic segmentation image instead of the original RGB image as the policy input.In this way,image noise can be effectively filtered out and important information will be kept for planning.Due to the limited computing resource of UGV and the real-time planning requirement,we choose a lightweight semantic segmentation model and design a teacher-student model to improve the generalization property.For lidar data,since there is no essential difference between simulation and real environments,we just make some standard processing.In policy module,we develop a lightweight multi-modal data fusion network,and use modal separation learning to alleviate the difficulty of policy learning from the multi-modal high-dimentional state space.Also,we introduce two-stage course learning to reduce the difficulty of policy learning in highly dynamic and complex scenarios and accelerate the policy training process,and use distributed training method to enable multi-agent interact with the environment in parallel to improve data sampling efficiency.In addition,we have designed a set of reward functions based on the local planning task to guide the policy learning.In experiments,we report the training curves of different stages for course learning in simulation.The UGV used is built on a Songling SCOUT robot chassis,equipped with hardware resources like NVIDIA Jetson TX2 computing resource kit,Raspberry Pi camera,Hokuyo UTM-30LX lidar,and software resources like LINUX operating system and ROS system.We compare our model with two ablation versions and some other state-of-the-art models in simulation and a real campus environment.The experimental results show that our model can narrow the gap between simulation and real environments more effectively,and the policy trained in simulation can be easily generalized to more static or dynamic environments.Finally,we test our model in the indoor corridor,and the performance is also superior to other models,which further shows that our proposed model can decouple perception and decision-making more efficiently.