Multi-3D-object Tracking By Fusing RGB And 3D-Lidar Data

Multi-object Tracking is one of the environmental perception algorithms.As an effective means of obtaining traffic information,it is considered to be a hot research topic in the field of computer vision.This dissertation conducts environmental perception by the Lidar data and the Visual data,besides,it researches multi-object tracking with multi-sensor from feature extraction and data association perspectives.To conduct a research on fusing information from multi-sensor.The method adopts feature information including 3D-location information of object,geometric features accessed by laser data and appearance features which based on visual data.It structures adjacent matrix by 3D-location information.The adjacent matrix can be conducted through the construction of the results of 3D-location information,and calculating the obtained adjacent matrix by The KM(Kuhn-Munkras)algorithm,then the result of the data association can be reached.Besides,Feature comparison module,particularly appearance features and geometric features,corrects the error(identity switch)caused by KM algorithm.To carry out a research on multi-object tracking based on object information prediction.This method is accomplished by two kinds of feature information,the one is trajectory prediction resulted from Siamese network and another is 3D-location information from detector.Meanwhile,these two kinds of information contribute to construct adjacent matrix.Then,the results of tracking association are to be calculated by KM algorithm.Additionally,the identity distribution regulation is established that are false negative and false positive to optimize the performance of multi-object tracking.This thesis is trained and tested on traffic driving dataset that KITTI Tracking dataset and BLVD dataset,and in following part,the evaluation metric such as MOTA and MOTP is used in the performance evaluation.The method of fusing multi-sensor information shows better performance in the comparison experiment with AB3 D tracking algorithm,which proves that it can deal with tracking problems more effectively.Moreover,the approach of object information prediction gains a better results in compare with self-base network and shows the validity of the method.