Research On Pedestrian Detection And Tracking In Crowded Scenes Based On Deep Learning

Pedestrian detection and tracking in crowded scenes is a very fundamental and important task in computer vision,which can be used in various practical applications.Therefore,realizing such an accurate and efficient pedestrian detection and tracking system in crowded scenes is of great significance.With the successful application of deep learning methods in computer vision in the recent years,researches on pedestrian detection and tracking have obtained considerable success.However,there are still many open problems in practical scenarios especially in crowded scenes.For example,a complex environment,illumination and weather conditions may affect pedestrian recognition;The nonuniform distribution of people in images may cause unexpected errors in crowd density estimation;The frequent occlusion between people may reduce the pedestrian detection accuracy and tracking robustness.To handle the aforementioned problems,this thesis proposes different algorithms,designs,and new models based on deep learning related methods to improve the performance of different sub-tasks,such as crowd density estimation,pedestrian detection,multiple pedestrian tracking,etc,realizing a more efficient and accurate pedestrian detection and tracking system in crowded scenes.The main work of this Ph D thesis is summarized as follows:(1)For the nonuniform distribution of people in crowd density estimation,the traditional multi-column density map estimation networks are simplified into a single column network with a feature fusion strategy.In addition,a semantic segmentation network is inserted to generate density masks which can reflect the density level.The masks are propagated into the density map estimation network together with the images for more accurate crowd density estimation.Experiments demonstrate that our approach can get the state-of-the-art results on several challenging datasets including our own new dataset.(2)To handle inaccurate small pedestrian detection and imprecise localization,this thesis designs an improved one-stage pedestrian detection framework based on the SSD method.The new algorithm has specific branch networks for classification and localization tasks,respectively.In the branch networks,on one hand,new feature fusion strategies and global attention mechanism are adopted to improve the detection performance.On the other hand,a new probability-based localization method is introduced and combined with traditional bounding box regression to improve the precision of pedestrian localization.Experimental results on Crowd Human dateset indicate that our method can perform much better than baseline methods.In addition,on general object detection datasets,MS COCO,our method also has impressive performance compared with other state-of-the-art object detection approaches.(3)For improving missing detections,false detections,and inaccurate detections,this thesis proposes a detection refinement method for multiple pedestrian tracking,which includes a detection refinement network(DRT-net)and an LSTM based motion model.DRT-net consists of an encoder-decoder backbone network and two side branches.The backbone network is trained to generate human heatmaps,which can guide the side branches to predict more accurate locations.For the two branches,one is used to refine the detected boxes,while the other is responsible for predicting potentially occluded targets.The LSTM based motion model is designed to forecast the human trajectories so that it can recover the lost targets in occlusion areas.Compared with strong baseline methods,our method achieves significant improvements on publicly available MOT datasets.In addition,DRT generalizes well,i.e.it can be applied to any detector to improve their performance.(4)In view of the unstable long-term tracking and frequent identity switches for multiple pedestrian tracking,this thesis proposes two novel methods based on a simple message passing network(MPN)to address these limitations.First,it explores different integration methods and puts forward a new Io U guided function which can improve the robustness of long-term tracking.Second,a hierarchical strategy is introduced to construct a sparse graph,which allows to focus the training on difficult examples.In this way,it can avoid frequent identity switches when tracking in occlusion areas.Experimental results demonstrate that a simple online MPN with these two contributions can perform better than many other state-of-the-art methods.In addition,this association method can also improve the results of any private detection-based methods.