A HRI Method Based On Stereo Vision And Deep Learning For UAV

Traditionally,interacting with UAV(Unmanned Aerial Vehicle)required specialized instrument and well trained operators.In many cases,the instrument based interaction method has been an obstacle in UAV application.In order to reduce the difficulty interacting with UAV.We make usage of the binocular camera on UAV which originally used in obstacle avoidance for motion capture by using depth sensing method.By using deep learning method for motion recognition,we develop a high accuracy human-robot interacting method which is non instrument based.This method reduces the interacting difficulty and has a great sense in experience of interacting with UAV.In our research,we finished following works:We set up an UAV platform for experimental usage.The platform includes an milticopter and an embedded signal processing platform equipped with NVidia Tegra K1 processor.We wrote an API which allow us control UAV from embedded signal processing platform.This has been the basis of UAV control,navigation and other further study.We designed and realize an interacting method with UAV based on stereo vision and deep learning.Firstly,tracking the people who was allowed to control the UAV and spilt it out.We got depth image which contained both the people and the background.We filtered out the background by normalizing and threadholding the depth image.Secondly,we overlay a series differential depth image.These image is colored by mapping the color and the depth image in HSV color space according the time of image captured to generate a colored texture image which including time and space information at the same time.Finally,we classify the colored texture image using deep learning method and recognized the gesture.We trained the neural network offline and executing the image classification online as the training of neural network required power computer.Finally,we built a data set containing four commanding gesture and a noncommanding gesture.We trained the neural network using this data set and prove the proposed classification method.The proposed method is robust for both indoor and outdoor situation and is effective in 10 meters.Make significant sense to the popularization of UAV and extend its application field.