Research On Object Tracking And Trajectory Prediction In Complex Traffic Environment Based On Deep Learning

Toward the future,electric vehicle autonomous driving is an important way to solve road traffic safety,ease traffic congestion,and improve driver comfort.Around the world,academia and industry have paid more attention to develop autonomous driving systems,and have achieved certain results.Autonomous driving systems are generally divided into three parts: environmental perception,path planning,and decision control.Realizing accurate perception of road traffic environment is of fundamental importance for autonomous driving systems.Vision sensor-based environmental perception is one of the main research directions at present due to its low price and wide adaptability.In the face of multiple visual attributes,traditional machine learning has been difficult to completely solve these issues.Deep learning has achieved good performance in various fields of computer vision due to its powerful representation capabilities.Visual perception algorithms based on deep learning are one of the most promising directions at present.This Ph.D.dissertation is based on the project of National Key Research and Development Plan named Research on the Key Issues of Sensing Environment,Dynamic Decision-making and Controlling for Intelligent Electric Vehicle(No.2016YFB0100900),and the project of National Key Research and Development Plan,Research and demonstration of key technologies for electric autonomous vehicles(No.2017YFB0102600).Considering the current spatio-temporal perception algorithms are insufficiently studied,the autonomous driving environment perception scheme is divided into single object tracking,multiple object tracking and trajectory prediction.The focus is on visual sensors based on deep learning methods to obtain the types and positions and movement path of targets in complex traffic environments.By analyzing the current status of relevant research at home and abroad,the problems and shortcomings in the current related research are summarized,and the main contents of this paper are determined.The main contents of this paper include a few aspects:1.A novel end-to-end realtime Siamese convolutional neural network single object tracking algorithm was proposed.Siamese networks have achieved great success in both accuracy and speed for visual tracking tasks.These Siamese trackers share a similar framework in which each tracker consists of two network branches for exploring semantic information.However,the performance of Siamese trackers is limited by an insufficient semantic template and an unsatisfactory updating strategy.To tackle these problems,we propose a manifold Siamese network for visual tracking that can simultaneously utilize semantic and geometric information.A manifold sample pool is constructed to exploit the manifold structure of image object sequences.This sample pool is dynamically learned via a fast Gaussian mixture model(GMM).After obtaining a manifold sample template,we design a deep architecture based on a correlation filter(CF)network and append a novel manifold feature branch.The network remains fully convolutional and can train a template to discriminate exemplar image and arbitrarily size search image.Then,a triplet occlusion score function cooperates with an effective update method that is established to prevent model drift.Extensive experiments show that the proposed tracking algorithm performs favorably compared with the state-ofthe-art methods on three standard benchmark datasets at a high framerate,which is very suitable for autonomous driving.2.A novel dual attention multiple object tracking algorithm was proposed.Existing methods mainly ignore prior information from real traffic scenes.In this paper,we propose a novel multiple object tracking algorithm that considers traffic safety for vulnerable road users.The proposed method integrates two attention modules with a novel detection refinement strategy.Since skilled drivers pay more attention to pedestrians and cyclists,we employ a saliency detection method to extract scene attention region.Then,a detection refinement strategy achieved a good tradeoff between parallel single object trackers(SOT)and detection results.Channel attention can mine the most useful feature channel for traffic road users.In the end,we operate our method on the popular MOT 17 benchmark in comparison with other high-level MOT algorithms.The tracking results show that the proposed dual attention network achieves state-of-theart performance.3.A novel driver perspective trajectory prediction was proposed.Many methods have been proposed to focuses on bird-view trajectory prediction for intelligent monitoring yet driver perspective is still unnoticed.Driver perspective trajectory prediction is a vital component for autonomous vehicles environment perception,which remains a challenging problem due to the high density,frequent occlusions and the randomicity of short-term path.In this paper,we propose a novel dual attention based LSTM network for driver perspective trajectory prediction.A novel LSTM module was proposed to capture historical information interaction between current object and surrounding objects.Furthermore,we derive a graph convolutional attention(GAT)module to model the spatial interactions between object groups.Then,we presented a temporal attention module to estimate the importance of object's behavior fragment.Finally,an end-to-end framework was composed by above three components.The quantitative and qualitative experiments on DiDi dataset proved that our algorithm can achieve competitive than state-of-the-art methods.