Research On Object Tracking And Trajectory Prediction Of Intelligent Vehicle Based On Vision And Radar Fusion

Intelligent driving technology has great potential in improving traffic safety,traffic efficiency and driving experience.It is a hot spot in the field of intelligent transportation at present and in the future.The intelligent driving system consists of three major software and hardware components: environment perception,decision planning and motion control.Environment perception understands the surrounding driving environment information and the vehicle's own state through sensors,which is an important prerequisite for intelligent driving vehicles.In a dynamic scene where the target-vehicle is occluded for a long time or closes to multiple similar vehicles,the accuracy of multi-object tracking and trajectory prediction will be significantly reduced,which affects the safety of intelligent driving vehicles.To solve this problem,with the Fund of the National Key Research and Development Program of China-“Theory of Human Machine Interaction in Intelligent Electric Vehicle”(No.2016 YFB0100904),the National Natural Science Foundation of China-“Research on Integrated Modelling and Control of Intelligent Electric Vehicles”(No.U1564211)National Key Research and Development Program of China-“Research on Simulation Environment Modelling and Hardware-In-the-Loop Testing Technology for Autonomous Vehicle”(No.2018YFB0105103),and the School-Enterprise Cooperation Project-“Development and Application of Autonomous Intelligent Driving System for Dongfeng Passenger Vehicle”,we carry out research on the object tracking and trajectory prediction of intelligent vehicles based on vision and radar.The main research work is as follows:(1)The IBN network structure and the SKNet network that adaptively adjusts its receptive field were introduced into the feature extraction network and a stacked hourglass network was used to build a predictive key point module,which realized lane detection based on key point instance segmentation.This solved the decrease of lane detection accuracy caused by changes in image appearance and the used of fixed-size convolution kernels to extract lane features.In the lane fitting and tracking part,firstly,the similar points outputted by the lane detection algorithm were clustered based on the DBSCAN algorithm,and the RANSAC algorithm was used to fit the parabolic model,and then the lane was tracked based on the Kalman filter algorithm.The algorithm was verified quantitatively and qualitatively on the Tu Simple data set and the vehicle experimental platform,respectively.(2)A visual three-dimensional detection and tracking algorithm framework based on deep learning was established.A dilated feature pyramid network structure for effectively extracting multi-scale features was proposed,so that each feature map had both deep semantic information and shallow edge information.It solved the problem of low detection accuracy of small objects when multi-scale object detection based on traditional feature pyramids network.On the basis of the feature regression network predicting the vehicle dimension and yaw angle,the object position coordinates were solved by using the constraint relationship between the detection frames.Built the prediction long short-term memory to model the object motion state,and predicted the position of the object in the next frame to obtain the object motion characteristics.The objects were sorted based on distance,then a linear model was used to predict the position of the occluded object,which reduced the mismatch caused by occlusion.According to the feature similarity and geometric similarity,the similarity between the objects could be calculated,and finally the weighted two-part matching algorithm was used for data association.Based on the feature similarity and geometric similarity,the similarity between the objects was calculated,and the weighted two-part matching algorithm was used for data association.(3)On the basis of the previous research,a framework of vision and radar distributed information fusion algorithm was proposed.The vehicle motion state estimation model was built,and motion compensation was performed of the radar object motion state on the basis of obtaining the lateral velocity of the ego-vehicle.For straight and curved road conditions,models were established to identify the positional relationship of the lane where the targetvehicle was located.In the radar object tracking process,the low accuracy and even divergence of non-linear filtering object tracking were caused by the unknown and time-varying of measurement noise.In this paper,based on the Square-root cubature Kalman filter,the SageHusa noise statistic estimator and the weighted index of fading memory were combined to derive the time-varying noise statistic estimator suitable for nonlinear systems.The target data association was achieved by using elliptical nearest neighbor data association algorithm in this paper.The motion state of the target-vehicle was divided into stationary,moving,oncoming,start-stop and unclassified.A method of classifying the motion state of the target-vehicle based on time window is proposed by analyzing the transfer mechanism of the motion state of the target-vehicle.On the basis of sequence track association,the track state after visual and radar information fusion was estimated by the convex combination fusion algorithm.The principle and calibration process of time synchronization and space alignment of vision and radar were introduced in detail.The algorithm was verified by using an experimental vehicle.(4)Based on the object tracking information obtained by vision and radar,a GPII-GRU trajectory prediction algorithm framework based on graph neural network and cyclic neural network was proposed.This paper proposed a solution from pairnorm and network architecture adjustment to the problem of smoothness in the process of using stacked multilayer graph neural networks to extract interactive features between traffic vehicles.Firstly,the pairnorm was introduced into the graph neural network without changing the network architecture and adding additional parameters.Then,a residual network model with initial residuals and identity mapping was introduced to the stacked multi-layer graph neural network modules.To a certain extent,it solved the over-smoothing problem of overlay multi-layer graph neural network layers.On the basis of obtaining the interactive features between traffic vehicles,the Bi-GRU encoder in the Seq2 Seq prediction model was used to extract the historical sequence trajectory features,and then the attention mechanism and the GRU decoder were used to predict the future trajectory of each traffic vehicle.The GPII-GRU trajectory prediction algorithm proposed in this paper were verified on the Apollo Scape Trajectory data set.The experimental results showed that the average displacement error and the final displacement error,which measure the accuracy of trajectory prediction performance indicators,had been significantly improved.