Research On End-to-end Autonomous Driving Based On Deep Neural Network

The study of autonomous driving is one of the most promising researches in this century.The key idea of studying self-driving is to find an ideal map between raw sensor data and driving behavior.Common self-driving systems can be categorized into two major paradigms: mediated perception approaches and behavior reflex approaches.The advantage of these approaches is that researchers can construct systems more simply,while the disadvantages are the complexity of the system and the high latency.However,in an end to end driving system,the driving behaviors are directly given after feeding the raw sensor data.So the end to end system is much simpler than the two common approaches and it is important for building a robust system with low time delay.Recently deep convolutional network has been widely used in computer vision field.It can dig information from large scale data.In this paper,we proposed some methods to learn the map between sensor data and driving behaviors.We test our methods in a dataset containing images from a HD camera,high quality point clouds from a Hesai laser and standard drivers' behaviors.The main contents of this paper are as follows:(1)We designed an end to end framework based on image data and convolutional network.Though some researchers had studied end to end self-driving network,most of their networks are either too simple or took too much memory of GPU.In this work,we built a deeper network which takes less memory.We trained and tested our network on our dataset and proved the possibility of learning the desired map.(2)We proposed a framework to use generative adversarial network to produce adversarial examples.Adversarial examples refer to those generated instances with small perturbation on source ones which are misclassified by the trained model.The generative adversarial network framework can produce images if the training policy is based on the game theory.In this paper,we proposed to combine them and use the generative model to produce adversarial examples rapidly.(3)A framework using point clouds as input are designed.Before,end to end self-driving systems only takes image as input.Due to the limitation of filed angle,cameras cannot capture the road in some situation,while the point clouds contain data of every angle.The point clouds are unordered data,so max pooling is used to address the problem.Skip layers are also used to collect feature of different dimensions.Experiments showed that the framework using point clouds as input achieved smaller average error and gave a better performance.(4)A framework based on fusion data is proposed.A projection can be made from point clouds to image array with the help of calibration parameter.Based on that projection,we create images whose pixel values are no longer RGB values but spatial coordinates of points.We stacked it with camera images and propose a convolutional network to learn the map between the stacked data and driving behaviors.The network based on fusion data achieved higher accuracy than the two former networks.But considering average error,it only beats the network based on image.