Deep Learning Based Obstacle Avoidance For Multi-rotor Drones Using Onboard Stereo Vision

In recent years,with the development of the society and the advancing of technology,Unmanned Arial Vehicles have attracted a lot of attention and are developing rapidly.Among them,the multi-rotor drones are widely put in use in many fields such as individual surveillance,topographic mapping,power line patrol,agriculture,pollution detection and so on.This happens because those multi-rotor drones usually have a lower cost,smaller size,lighter weight and the ability to hover in the air.With the UAV becoming more and more widely used in both military and civilian area,the requirements for UAV are becoming higher and higher.Especially when conducting tasks in low altitude,the ability to sense and avoid obstacles for UAV are required.This paper does research on the UAV's ability to sense and avoid obstacles,using the convolutional neural networks,which is a technique in deep learning area,to process the images obtained by UAV.Through this,we can get what kind of obstacles are contained in the images and where they are in the images.Then we use stereo vision method to obtain the relative position between the UAV and obstacles.With those information,we can make the UAV avoid the obstacles successfully and manage to finish their works.The main works of this paper are listed as below.(1)This paper contrasts two types of methods on detecting the obstacles in the UAV's view.One detecting method is based on image feature points and the other one rely on the convolutional neural networks.In this part,a conclusion is made that the deep learning based obstacle detecting method is of wider applicability.Using deep learning based obstacle detecting method makes the UAV learn much more information from pictures,and improves UAV's understanding of the environment.The proposed method has a reasonable time cost running on GPU and can meet the time requirement.(2)This paper proposes a method to estimate the depth of detected obstacles based on the obstacle detecting result.Compared to the traditional way,the proposed method does not need large amount of computation or complex matching process.Experiment show that the proposed method is correct and is capable to put in use.(3)In this paper,the proposed method,a deep learning based obstacle avoidance algorithm for droness using binocular vision,is realized based on the Robots Operating System.Based on the Gazebo platform,we build the simulation environment,verify the method in simulated ways.Then we build a Quadrotor using Pixhawk,ZED camera,TK1 processor and conduct the physical experiment to verify the proposed method.