Collision Avoidance For Indoor UAV Based On Deep Reinforcement Learning

UAV can replace human beings to accomplish many difficult tasks,and its autonomous navigation has always been a research problem in the field of drones.The effectiveness of traditional model-based drone navigation methods requires the drone itself to accurately model the surrounding environment information,which has led to the traditional drone navigation algorithm's ability to adapt to unfamiliar environments is greatly reduced.On the other hand,during the evolution of natural creatures,they have a strong ability to adapt to the uncertainty of the environment.Therefore,this article examines the adaptive navigation of drones in indoor environments from the perspective of biological adaptive reinforcement learning.The problem.Reinforcement learning-based drone navigation has two key issues: reinforcement learning strategy training and reinforcement learning strategy transfer.In particular,the sensors used in this work are monocular cameras,which is still a problem for indoor environments with pedestrians.This article studies these three issues in depth.Aiming at the problem of reinforcement learning policy training,this paper proposes an improved model of deep reinforcement learning model based on Deep Deterministic Policy Gradient(DDPG),in order to improve the adaptability of existing autonomous navigation strategies of drones to the environment and the learning speed.The model consists of three parts: first,using only lidar data as a state input to sense environmental information;second,designing a reasonable reward function to stimulate the strategy to learn faster and better;finally,designing reasonable actions Space makes drones make smooth decisions.After reinforcement learning training in a simulation environment,the indoor UAV equipped with a single-line lidar can perform stable obstacle avoidance navigation in a simulation environment,and also has a better adaptive ability in an unfamiliar simulation environment.Aiming at the problem of reinforcement learning policy transfer,a new framework based on cross-sensor migration learning is proposed to improve the migration effect of strategies trained in the simulation environment to the real world.This cross-sensor transfer learning framework,in the process of transfer learning,first uses only virtual single-line lidar as a sensor in the simulation environment,and uses DDPG deep reinforcement learning to train a stable primary obstacle avoidance strategy.Secondly,use the monocular camera and lidar to collect the visual and depth data sets in the real environment and bind them frame by frame,use the primary obstacle avoidance strategy to automatically label the real data set,and train to obtain the monocular visual obstacle avoidance strategy without lidar to realize cross-sensor transfer learning from virtual lidar to realistic monocular vision.Aiming at the problem of monocular navigation without depth information in a pedestrian environment indoor,an asynchronous deep neural network structure composed of YOLO v3-tiny network and Resnet18 network is proposed to realize the combination of planner and pedestrian information,and alleviate the excessive pedestrian difference The instability of the strategy makes it possible to effectively and stably avoid obstacles in the presence of pedestrians in the absence of depth information.In order to verify the effectiveness of reinforcement learning strategies,transfer learning,and parallel deep neural network structures.Finally,we carried out experiments on the real DJI Mavic drone.Experiments show that the physical drone can finally perform stable and efficient obstacle avoidance navigation in a realistic indoor unmanned corridor,an indoor human corridor,and an indoor environment with unstable light.