Visual Target Detection And Tracking In Anti-drone System: Algorithms And Implementation

The opening of low-altitude airspace has led to the vigorous development of the low-altitude aviation industry.With huge economic and social benefits,it also poses unprecedented challenges to the safety and control of low-altitude airspace,especially for the low,slow and small targets which are difficult to find,capture,handle and deal with.Typical representatives are gliders,delta wings,hot air balloons,tethered balloons,model airplanes,etc.,especially small multi-rotor drones.Due to low cost,simple operation,light weight and portability and dexterity,drones have been widely used in various fields.But the subsequent large illegal uses of small drones have posed great threats to national security,civil aviation traffic,social security and personal privacy.The study of anti-drone technology and system have become an important research area of acadimia,industry and military.Anti-drone monitoring technology is the basis and prerequisite of UAV countermeasures.Low-cost equipment,no radiation,good concealment,clear and intuitive monitoring results make visual monitoring become one of the most prevail anti-drone surveillance methods,which can realize early warning,detection,identification and tracking of drones.Based on the computer vision technology,this dissertation uses optical cameras as sensors to conduct in-depth research on anti-drone vision detection and tracking in different environments.The main work and contributions are summarized as follows:1.This dissertation proposes an adaptive switching spatiotemporal fusion detection method for remote flying drones in the airspace using electrical-optical cameras,which can enhance the contrast between the target and background as well as suppressing the noises and clutters simultaneously.For each incoming video frame,a dark-attentive interframe difference method and a row-column separate black-hat method are proposed to generate temporal feature maps(TFM)and spatial feature maps(SFM),respectively,in parallel.Inspired by the phenomenon that the features in TFMs and SFMs both go strong at the regions of the intended target while they do not at other regions where noises and clutters locate,an adaptive switching spatiotemporal fusion mechanism is designed to fuse the SFMs and TFMs,generating adaptive switching spatiotemporal feature maps(ASSTFM).Finally,an adaptive local threshold mechanism is used in ASSTFMs to segment the targets from backgrounds.The experiment results manifest that our method is superior to the other seven baseline methods and works more stably for different backgrounds and various types of drones.2.This dissertation proposes a data-driven support vector machine(SVM)based spatiotemporal feature fusion detection method for small low-contrast targets.A novel pixel-level feature,called a spatiotemporal profile,is designed to depict the discontinuity of each pixel in the spatial and temporal domains.Instead of the handcrafted feature fusion mechanisms in previous works,the labeled spatiotemporal profiles are used to train an SVM classifier to learn the spatiotemporal feature fusion mechanism automatically.To speed up detection for high-resolution videos,the serial SVM classification process on CPUs is reformed as parallel convolution operations on GPUs,which exhibits 1000+ times speedup in our real experiments.Finally,blob analysis is applied to generate final detection results.Elaborate experiments are conducted,and experiment results demonstrate that the proposed method can realize better detection performance for small low-contrast target.3.This dissertation proposes an integrated and effective real-time visual drone surveillance system using pan-tilt-zoom(PTZ)cameras.This system follows a novel framework that integrates a deep learning based detector and a fast motion adaptive kernelized correlation filter(KCF)tracker into a closed loop by an adaptive switching mechanism.The detector consists of a moving region proposals(MRPs)generator and a lightweight deep convolutional neural network.A search window relocation strategy based on Kalman filter is proposed to improve the fast motion handling ability of the KCF tracker.Finally,an adaptive switching mechanism based on the reliable detection result and the response map of the KCF tracker is proposed to achieve strong discriminative capability while maintaining the use of spatiotemporal continuity of the video frame.Both the offline and real-world experimental results demonstrate the effectiveness and real-time processing ability of the propsed system.4.This dissertation proposes a multi-camera collaborative counter-drone surveillance system to search,classify and track the flying drones,which solves the contradictory that a single camera cannot see widely and see targets clearly that are far away simultaneously.The remote drone system is based on a master-slave framework.The master overview camera with a wide field of view is responsible for monitoring the suspicious flying targets,which follows a multi-stage pipeline.After sky-ground segmentation,a spatiotemporal feature fusion detection method is proposed to detect the small suspicious targets in the sky.Then a multi-target tracking algorithm is used to generate their trajectories and a threat level assessment criteria combining the features of both the trajectory and appearance is designed to evaluate the trajectories’ threat.After sorting the suspicious targets in terms of their threat levels,the overview camera guides the pre-calibrated slave PTZ telephoto camera to quickly search,identify and track the targets using a YOLO v4 based drone detection method and a proportional-integral controller.In the meantime,the available PTZ camera is used to scan the near ground to detect the drones in short and middle ranges.Both the experiments and the field tests validate the effectiveness of our system.In the end,this dissertation concluds the thesis and discuss some possible future research directions.