Multiple Object Tracking Based On Target Association Optimization On Unmanned Aerial Vehicle

Multiple object tracking is an important research topic in the field of computer vision,and it is an indispensable part of computer video analysis system.It plays an important role in public security,ocean surveillance,port surveillance,indoor surveillance,group behavior analysis,driverless car driving,UAV automatic navigation and other tasks.As an important part of computer vision,it is of great significance to study more accurate multiple object tracking methods.At the same time,with the development of UAV technology in recent years,UAV has the advantages of small size,portability and flexibility,which can be applied to many industries such as safety,transportation,agriculture,etc.The UAV video has its own difficulties and challenges,so the research of multi-object tracking technology for UAV video is of great significance in practice.However,only a very few UAV datasets have been constructed so far and most datasets are limited to a specific task.Therefore,in order to verify the effectiveness of the tracking and detection algorithms,in this paper we construct a challenging UAV dataset,and we annotate about 80000 frames obtained from 10 hours of original videos,providing about 840,000 frames with more than 2,700 targets,and proposing new challenges,such as complex scene,high density,small target,camera motion,etc.At the same time,the state-of-art object detection and tracking methods are evaluated on our dataset,while the challenges brought by UAV videos are analyzed?With the development of object detection technology,the effect of object detection gets better.The multiple object tracking method based on data association has attracted more attention.In the existing under tracking-by-detection framework,tracking accuracy still could be improved the in the two stages,e.g.,object expression and data association allocation.When calculating the association relationship of each target,we need to select effective feature cues,calculate the similarity of objects in each cue,and fuse the similarity results of each cue.In order to get a more optimized relationship between the objects,from back-end to front-end,this paper puts forward three research points:Firstly,we propose a self-balance motion and appearance model for multiple object tracking in UAV to optimize the cue fusion.Most methods represent objects by their appearance and motion.The inference of association is mainly judged by the fusion of appearance similarity and motion consistency.However,the fusion weight of appearance and motion is often determined by subjective settings.We propose a self balancing method which combines appearance similarity and motion consistency.For each object in the scene,when its appearance or motion discrimination ability changes,the adaptive weight combining appearance similarity and motion consistency is generated to optimize the rationality of each part of cue fusion and enhance the stability of tracking.At the same time,we verify the effectiveness of our tracking method on the proposed UAV dataset.Secondly,we propose a conditional GAN based individual and global motion fusion for multiple object tracking to optimize the motion prediction.Different from the traditional video,the appearance information of the target object is not reliable because of the high flight altitude and motion mutation of the UAV.Motion analysis is of great significance for the association of multiple targets in the UAV video.We propose a conditional GAN based model to predict complex motions in UAV videos.We regard the target motion and UAV motion as individual motion and global motion respectively.A social LSTM network is used to estimate the individual motion of the object,the Siamese network is constructed to generate the global motion to reflect the view changes from UAVs,and the condition GAN network is developed to generate the final motion affinity.A large number of experiments have been carried out on the public UAV dataset.Compared with the state-of-art motion prediction method,this method has robustness and improved performance.Lastly,we propose a online multiple object tracking via association between objects with context cue in which novel clue is introduced.Most of the existing tracking detection methods are based on the appearance and motion of the target.However,the context information around the target has not been fully utilized.Taking full advantage of the context information,this paper proposes an EOCS model,which uses a new affinity and optimizes the bounding box by measuring the change of context background.We propose we propose a novel approach based on the so-called Context-aware Multi-task Siamese Network(CMSN)model that explores new cues in UAV videos by judging the consistency degree between objects and contexts,which makes the tracking algorithm more robust.A large number of experimental results on the public dataset prove the effectiveness of the proposed tracking method.