Deep Particle Correlation Filter For Real-Time And Robust Object Tracking

Visual object tracking is one of the core problems in the field of computer vision due to its numerous applications including pose estimation,robotics,human-computer interaction,unmanned driving and et al.Given the initial bounding box of an object in the first frame over the video sequences,the task of visual tracking is how to robustly estimate the object state in the following video frames.Despite many such trackers have been proposed in the past few years,real-time and robust visual tracking is still a challenging task in scenarios with in-plane and out-of-plane rotations,occlusion,deformation,motion blur,fast motion,illumination changes,background clutter,scale variations and real-time application.Based on an in-depth analysis of particle filter frameworks,correlation filter theories,and deep learning technologies in visual tracking,this paper focuses on the deep particle correlation filter for high-performance visual tracking in open environments with various challenges and real-time applications from the four aspects of scale estimation,feature description,sampling strategy and running speed.The main contributions of this paper are as follows:1)Accelerated particle filter for real-time visual tracking with decision fusion.Since correlation filters treat prediction regions in current frames as training data,they can be contaminated by detected and accumulated incorrect results which cause model drift.Particle filters employ rich prediction candidate regions to estimate the target state,but suffer when the environments are complex throughout an image sequence.For the above problems,we propose an innovative accelerated particle filter for real-time visual tracking.We integrate multiple correlation filters as observation models into the particle filter framework.The dense-sampling strategy of correlation filters can reduce particles to be sampled in particle filters.Rich candidate regions in particle filters can alleviate incorrect predictions of correlation filters.Besides,we propose a decision fusion strategy to integrate many different types of features.Particle filters and decision fusion strategies with multi-features can produce more accurate predictions to alleviate model drift in correlation filters.Extensive experiments demonstrate the proposed tracker is very promising compared with state-of-the-art trackers while operating at 89 frames-per-second.2)Lightweight particle filter for accurate scale estimation robust visual tracking.Aiming at the problem of deep convolution features-based robust object tracking in scenarios with large-scale variations and high computations caused by repeatedly extracting features of deep convolution networks in the particle filter,we first propose a lightweight particle filter algorithm based on the in-depth analysis of the particle correlation filter.It can not only achieve fast and accurate scale estimation,but also can be integrated into any tracker without scale estimation.Furthermore,we propose an effective generic parallelization framework,which not only avoids the high complexity problem caused by the deep particle sampling,but also efficiently uses the robust advantages of deep convolution features while takes the fast and accurate scale estimation of the lightweight particle filter into account.Comparison experiments with several state-of-the-art trackers show our lightweight particle filter not only outperforms them on the tracking performance,but also operates at 60 frames-per-second.Compared with the baseline tracking method,our tracking approach based on the generic parallelization framework achieve the performance improvement in various challenging scenarios,which further validates the importance of accurate scale estimation.Finally,our tracking approach based on the generic parallelization framework and deep convolutional features also performs favorably against state-of-the-art tracking methods.3)Robust visual tracking via hierarchical particle filter and ensemble deep features.Particle filters need to draw a large number of samples to ensure the accuracy of target state estimation,but their tracking efficiency typically suffers especially when deep convolutional features are applied in extracting samples.Deep convolutional features have been introduced into visual tracking,but their powerful target appearance representations have not been fully explored.For the above problems,we propose to elegantly exploit deep convolutional features with few particles in a novel hierarchical particle filter,which breaks the standard particle filter down into two constituent particle layers,namely,particle translation layer and particle scale layer.The particle translation layer focuses on the object location with the deep convolutional features capturing semantics,while the particle scale layer pays attention to large-scale variations with the lightweight hand-crafted features handling spatial details of the object size.Moreover,an efficient ensemble method is proposed to help explore deeper features of different convolutional neural networks with more semantics in the particle translation layer.Extensive experiments on public tracking datasets demonstrate that the proposed method performs favorably against a number of state-of-the-art trackers,and achieves 93.4%and 69.3%tracking results with precision and success scores on the OTB-2013 dataset.4)Particle scale space for real-time and robust visual tracking with correlation operators.For the problem of fixed scale strides and the lack of temporal scale information in the hand-crafted scale space adopted by most scale estimation methods,we propose the correlation operator-based particle scale space exploiting the temporal scale information to adaptively generate scale strides,which not only should be robust to the large-scale changes of target appearance but also takes the computational efficiency into account for real-time applications.Besides,our proposed particle scale space is generic and can be extended to many siamese network and correlation filter-based trackers with hand-crafted scale space and correlation operators.Finally,in order to verify the generality and effectiveness of the proposed method,we propose a series of particle scale space trackers and test them on large-scale public datasets.Extensive experiments show that the proposed particle scale space-based trackers are not only better than the baseline trackers with hand-crafted scale space,but also perform favorably against state-of-the-art tracking approaches.