Research On Multiple Object Tracking Based On The Joint-Detection-and-Tracking Paradigm

Multiple Object Tracking(MOT)is one of the hottest research areas in the field of computer vision and has attracted considerable attention from both academia and industry recently.This task aims to locate all interested objects in the video sequence,give different objects their respective identities,and maintain the identities of objects in different frames.As one of the components of some high-level tasks,MOT has high academic value,and it has great practical and economic value in fields such as autopilot,intelligent security,and unmanned aerial vehicle applications.At present,the mainstream detection-based online MOT algorithms are divided into two paradigms: one-shot and two-stage,also known as the Joint-Detectionand-Tracking(JDT)and Tracking-By-Detection paradigm.Although the JDT methods have achieved excellent performance,there are still many key issues restricting the improvement of the algorithm performance,such as the incompatibility between sub-tasks in this paradigm,the lack of time information caused by single frame input,mutual interference between objects,the restriction of detector performance and so on.This paper explores some of the above key issues,and the main contents are summarized as follows:(1)This paper proposes multi-layer channel enhanced feature rearrangement for MOT.In the JDT paradigm,different subtasks are merged into a unified network model.According to the model performance under different training modes,the existence of the sub-task incompatibility problem is verified.Through the analysis of the problem,this paper proposes the Channel Enhanced Feature Rearrange(CEFR)module to relieve the optimization conflict during training,and then alleviate the sub-task incompatibility problem.The CEFR module will model the channel information globally,so that each sub-task can learn the information suitable for itself from the modeled vector during training.And then the module further enhances the information interaction between channels through the channel shuffle operation.Finally,multiply the residual branch element by element to obtain the final task-specific features.In order to increase the difference between subtasks and balance the difference and sharing,this paper further proposes the Multi-layer Channel Enhanced Feature Rearrange(MCEFR)architecture,which rearranges the features in the backbone network.The feature rearrangement of different scales can help the network to be more robust in the face of the change in object size and improve the representation ability of the model.The proposed method has achieved competitive results on MOT16,MOT17 and MOT20 test sets.(2)This paper proposes the dual-dimensional inter-frame information utilization architecture for MOT,supplying time information for the JDT paradigm of single frame input without changing the output result of the model.The proposed architecture consists of the Dualdimension Global Awareness(DGA)module and the Horizontal and Vertical Relocation(HVR)module.The DGA module is responsible for extracting the useful information in the previous frame from the spatial dimension and the channel dimension as the supplementary information of the current frame branch,and outputting the feature map for different sub-tasks.The performance of using the DGA module alone is not ideal,so the HVR module is required to relocate the object at the fine-grained level on the feature map with the confused time information from the vertical and horizontal perspectives.In addition,in order to reduce the influence of two-frame image input on the inference speed,the model also uses the inference technique of feature reusing.The experiments on the MOT17 validation set proved that only the combination of the DGA module and the HVR module would be beneficial to the model,and the experiments on the MOT16,MOT17 and MOT20 test sets proved the effectiveness of the proposed architecture.(3)This paper proposes a data association strategy based on trajectory quantification,which can be added to most detection-based MOT algorithms in a plug-and-play way.In order to reduce the interference between the objects in the video sequence,this strategy shows the quality of the trajectory explicitly.The quality of a trajectory at a certain time is determined by its quality at the previous time and the matching result at the current time.Different updating functions are adopted according to different matching conditions.Based on the quantified trajectory score,this paper proposes the trajectory life cycle management(TLCM)strategy,hierarchical trajectory matching(HTM)strategy and missing trajectory prediction(MTP).TLCM defines the initialization and termination of trajectories from the perspective of quantified trajectory score.Therefore,high-quality trajectories can participate more in matching while low-quality trajectories can be terminated earlier.HTM matches trajectories of different quality separately to reduce the interference of low-quality trajectories on high-quality trajectories and improve the possibility of low-quality trajectories being connected during matching.MTP removes more false negative samples and improves the performance of the tracker at the cost of a small amount of false positive samples.The data association strategy based on trajectory quantification is applied to other trackers and tested on the MOT17 validation set,which proves the universality of the strategy.Meanwhile,the proposed strategy is compared with the excellent MOT algorithm in recent years on the MOT16,MOT17 and MOT20 test sets,which proves that it can improve the tracking accuracy and make the object’s identity more robust.