Research On Multiple Object Detection And Tracking Problems In Complex Traffic Scenes

Multiple object detection and tracking are among one of the core problems of computer vision.It is the basis of many applications such as video analysis,behavior recognition,and abnormal event detection,consisting of detection,feature modeling,and affinity association.Although in recent years,with the development of deep learning,a large number of excellent methods have emerged in the field of multiple object detection and tracking.There are still many problems.Among these problems,there are three critical problems needed to be solved,namely,1)under particular application scene,hardware constraints make it difficult to achieve fast object detection and tracking,2)separation of feature modeling and affinity association results in poorly object feature modeling and association for multiple objects,and 3)the separation of the detection and tracking can result in the complexity of the tracking and detection problems.In response to these obstacles,this thesis focus on multiple object detection and tracking problems from three aspects,as follows:1)Pedestrian detection and tracking algorithm based on RGB-D camera environment is proposed,and a large-scale pedestrian counting datasets(PCDS)are published in this thesis.Furthermore,the proposed algorithm is also applied to the problem of bus pedestrian counting combined with a pedestrian counting algorithm based on the RGB-D camera.These algorithms include explicitly an automatic calibration method of the RGB-D camera,a pedestrian head detection method based on the depth map,and a pedestrian tracking and trajectory analysis method.Besides,this thesis also publishes a pedestrian counting dataset.The experimental results show that the proposed automatic calibration method has strong robustness and can meet the requirements of application accuracy.The pedestrian head detection method proposed in this thesis can meet real-time needs.The pedestrian tracking and trajectory analysis methods proposed in this thesis can accurately count the pedestrian in real-time,even under limited hardware conditions.2)An end-to-end deep learning network for both feature modeling and matching association is proposed.Target To tackle the problems caused by the separation between feature modeling and affinity association,an end-to-end network(Depth Affinity Network)is designed for modeling the objects' features and performing multiple object affinity associations.Besides,this thesis also employs the Deep Affinity Network into a tracking method.The experimental results show that the deep affinity network-based tracking method introduced in this thesis can accurately track the objects and solve most of the target loss caused by occlusion.Our technique is evaluated on prevalent multiple object tracking challenges MOT15,MOT17,and UADETRAC.Comprehensive benchmarking under twelve evaluation metrics demonstrates that our approach is among the best performing techniques on the leader board for these challenges.3)A tracker based on detection and tracking integrated network,Single Shot Detector and Tracker Network(SSDTN),is proposed,and a large-scale Awesome Multiple Object Tracking Dataset(AMOTD)is published.This thesis creates a detection and tracking integrated network for eliminating the division of multiple object detection and multiple object tracking.The network estimates motion equations,classification,and visibility for all targets in various video frames.Besides,based on this network,this thesis applies the SSDTN into a tracking method.Moreover,multiple large-scale objects tracking dataset is released,which is more than 100 times of UA-DETRAC dataset.Experiments show that the network can accurately estimate the objects' motion equation.Furthermore,the methods can jointly perform the detection and tracking at 116 fps.This research solves the mentioned problems associated with multiple object detection and tracking.The thesis introduces the detection and tracking methods in a specific scene by utilizing depth images.Combined with the trajectory analysis algorithm,the proposed methods can also automatically counting the pedestrian passing through the bus.The pedestrian counting accuracy can reach more than 85%,and its speed can reach 45 fps.Target to improve the accuracy of MOT,the thesis also proposes a Deep Affinity Network(DAN).Based on this network,the DAN tracker is designed.The tracker can achieve high scores in both the MOT17 and the UA-DETRAC competitions,and the speed can reach more than 6fps.Aiming to improve the speed of multiple object detection and tracking,the single-shot object detection & tracking network is also proposed.Based on this network,a tracker is further designed.The speed of this tracker can reach 116 fps under the premise of ensuring tracking accuracy.Besides,this thesis also published two large-scale datasets,the People Counting DataSet(PCDS)and the Awesome Multiple Objects Tracking Dataset(AMOTD),which can be used in the field of people counting and multiple object tracking.The above work is of great significance for the research of multiple object detection and tracking problems.